{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [DeepSphere]: a spherical convolutional neural network\n",
    "[DeepSphere]: https://github.com/SwissDataScienceCenter/DeepSphere\n",
    "\n",
    "[Nathanaël Perraudin](https://perraudin.info), [Michaël Defferrard](http://deff.ch), Tomasz Kacprzak, Raphael Sgier\n",
    "\n",
    "# Demo: part of sphere classification\n",
    "\n",
    "This demo uses the whole datataset, smoothing, and the addition of noise.\n",
    "\n",
    "**You need a private dataset to execute this notebook.**\n",
    "See the [README](https://github.com/SwissDataScienceCenter/DeepSphere/tree/master#reproducing-the-results-of-the-paper).\n",
    "But you can use it with your own data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 0.1 Load packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autoreload extension is already loaded. To reload it, use:\n",
      "  %reload_ext autoreload\n"
     ]
    }
   ],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import shutil\n",
    "import sys\n",
    "\n",
    "# Run on first GPU.\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"0\"\n",
    "sys.path.append('../..')\n",
    "\n",
    "# To get the CUDA profiler (do it on the CLI before starting jupyter):\n",
    "# export LD_LIBRARY_PATH=/usr/local/cuda-9.0/extras/CUPTI/lib64\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from deepsphere import models, experiment_helper, plot, utils\n",
    "from deepsphere.data import LabeledDatasetWithNoise, LabeledDataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.rcParams['figure.figsize'] = (17, 5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 0.2 Definition of the parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### A) Non tunable parameters\n",
    "These parameters are fixed or the preprocessing script has to be modified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "Nside = 1024\n",
    "sigma = 3\n",
    "data_path = '/mnt/scratch/lts2/mdeff/deepsphere/data/same_psd/'\n",
    "# data_path = 'data/same_psd/'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### B) Tunable parameters\n",
    "These parameters can be changed.\n",
    "\n",
    "We choose to work in the noiseless setting by setting `sigma_noise = 0`. This allows this notebook to run an acceptable time. In the noisy case, the training of the network needs considerably more iterations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "order = 1  # 1,2,4,8 correspond to 12,48,192,768 parts of the sphere.\n",
    "sigma_noise = 2  # Amount of noise for the experiment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1 Data preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1 Data download\n",
    "Set `download` to `True` to download the dataset from zenodo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "download = False\n",
    "if download:\n",
    "    %run -i 'download.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Data preprocessing\n",
    "Apply the preprocessing steps.\n",
    "1. Remove the mean of the maps\n",
    "2. Smooth with a radius of 3 arcmin. (`sigma` parameter)\n",
    "\n",
    "Set `preprocess` to `True` to execute the preprocessing script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "preprocess = False\n",
    "if preprocess:\n",
    "    %run -i 'data_preprocess.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us display the resulting PSDs of the preprocessed data. We pre-computed the PSDs for faster execution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "compute = False\n",
    "if compute:\n",
    "    psd = experiment_helper.psd\n",
    "    data_path = 'data/same_psd/'\n",
    "    ds1 = np.load(data_path+'smoothed_class1_sigma{}.npz'.format(sigma))['arr_0']\n",
    "    ds2 = np.load(data_path+'smoothed_class2_sigma{}.npz'.format(sigma))['arr_0']\n",
    "    psds_img1 = [psd(img) for img in ds1]\n",
    "    psds_img2 = [psd(img) for img in ds2]\n",
    "    np.savez('results/psd_data_sigma{}'.format(sigma), psd_class1=psds_img1, psd_class2=psds_img2)\n",
    "else:\n",
    "    psds_img1 = np.load('results/psd_data_sigma{}.npz'.format(sigma))['psd_class1']\n",
    "    psds_img2 = np.load('results/psd_data_sigma{}.npz'.format(sigma))['psd_class2']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The PSD of the two classes is almost indistinguishable. \n",
    "\n",
    "Spoiler Alert! This is the reason why PSD features are not good enough to classify the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ell = np.arange(psds_img1.shape[1])\n",
    "\n",
    "plot.plot_with_std(ell,np.stack(psds_img1)*ell*(ell+1), label='class 1, $\\Omega_m=0.31$, $\\sigma_8=0.82$, $h=0.7$', color='r')\n",
    "plot.plot_with_std(ell,np.stack(psds_img2)*ell*(ell+1), label='class 2, $\\Omega_m=0.26$, $\\sigma_8=0.91$, $h=0.7$', color='b')\n",
    "plt.legend(fontsize=16);\n",
    "plt.xlim([11, np.max(ell)])\n",
    "plt.ylim([1e-6, 5e-4])\n",
    "plt.yscale('log')\n",
    "plt.xscale('log')\n",
    "plt.xlabel('$\\ell$: spherical harmonic index', fontsize=18)\n",
    "plt.ylabel('$C_\\ell \\cdot \\ell \\cdot (\\ell+1)$', fontsize=18)\n",
    "plt.title('Power Spectrum Density, 3-arcmin smoothing, noiseless, Nside=1024', fontsize=18);\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 Data loading\n",
    "The following functions will\n",
    "1. Load the preprocessed data\n",
    "2. Create samples by dividing the complete spheres in patches (based on healpix sampling). See the function `hp_split` of `experiment_helper.py` for more specific informations.\n",
    "\n",
    "The function that load the testing data will additionally add the noise to the sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 149,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_raw_train, labels_raw_train, x_raw_std = experiment_helper.get_training_data(sigma, order, data_path=data_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_raw_test, labels_test, _ = experiment_helper.get_testing_data(sigma, order, sigma_noise, x_raw_std, data_path=data_path[:-9])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Non normalized Laplacian\n",
    "* degree hist\n",
    "* less neighbours 20"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "Gold = utils.healpix_graph(1024, True, 'normalized', indexes=utils.nside2indexes([1024], 1)[0], new=False)\n",
    "Gnew = utils.healpix_graph(1024, True, 'normalized', indexes=utils.nside2indexes([1024], 1)[0], new=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'weighted degree distribution: DeepSphere V1')"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA/MAAAFACAYAAADwPSxRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAAIABJREFUeJzt3X+8XVV95//Xm1+W+guUiHwJNYzGKugYJcW0dBwFCwE7RS220FFSSpvWwRntlFHQbwWptNpqmdIqLQoSLBapysAolqb88EdHgYDITy0RU4kgBIMIg2LBz/yx15XD4dx7zw1Jbnbu6/l47Mc5Z+219157n3VP8j5773VSVUiSJEmSpP7YZrYbIEmSJEmSZsYwL0mSJElSzxjmJUmSJEnqGcO8JEmSJEk9Y5iXJEmSJKlnDPOSJEmSJPWMYV6S5rgkJya5YYbLXJ7krzZVm0Zs79NJzprhMocl2ep+fzXJsUnWDLye8fs3g22tSXLswOtKctgm2tYm24+5KsnL23u2y2y3RZK08RnmJUnvA/7jxl7pcBDUJjP2+7cB4e7ngA9ucMtGt2FBa8PioVmbpB+O2aY1rU2V5IdJbktyfpL/NBvtaW16UZILknyntelbST6Z5Fmz1abHK8l1ST48ybxD2vF/bnv9F0lWtX1fs1kbKkk9YZiXpDmuqu6vqu/Odjv6KskOs7n9TfH+TexTVa2rqgc25ronswX0w5OA3YDnAocDa4Dzk/zl5m5IknnAJcD9wKuA5wFvAL4BPGVzt2fY4+jzZwC/nuSJI+b9FvCFqvqX9nobYAVw9gZuS5K2eoZ5SeqRJAcnuS/Jdu31wnY267SBOicnWTnweq8kn2nL3ZXk75I8c2D+oy5vTrJdklOS3NOmU5KcluTyoeZsk+SPk9zd1vu+JNu0dVwOPAv4s4kzngPr/4Ukn0vyQJJvt3U/ZWD+Tyc5K8n9Se5M8vYxj82RSf61rffTwK4j6vynJFe3s33fbMdqh4H5uya5MMkP2rqOSnJDkhMH6lSSY5J8Ksn/Bf54nOPc6hyV5Ka2/X9J8vsTx2yK/XprOzt7f5KzgScNzR9+/16Y5JIk329t+WqSVyRZAFzWqq1r+3FWW+by9j68L8k64J9b+airK57Z9vOBdoxeP7DtkWfd8+jL87/ZHq9q5ZdPsh/bJPnDdGfJH0xyfZJDR2zrV5OsbO25KckvTXU8p3BfVX2nqr5VVf9cVb8P/BfgTUleMbDd3ZOcO/D38ZkkC4f2d7p+tqbt79+29/U7Q8d5P2Bn4Kiqurqq1lTV56rqrVV1/dD+/0aSL7ZtfS3JgSP27UVJrmjHaFWSlwy1d7q/ycn6x1OTnN76+31tHcNXXAz6KLA98GtD258H/Arwk7P2VfVfq+ovgX9BkjSSYV6S+uULwE8BE/9hfjlwN/CKgTovBy4HSLIb8HngBmBf4JV0YfDCKULkscBvAr8NLKH7t+I3RtT7z8BDwC8AbwLeAvx6m/daYC2PnO3crbXnhcA/AhcCL2r1FgFnDqz3fcAvAb8KHAC8GHjZJG2lrfelwFnA6W19/7tte7DOQcA5wF8Be9OdCTyMFsabFXRfQuwPHAq8vr0edgJwEfBC4APjHOckv9O29U7g+cAfAG+jC4yT7devAe9u23sJ8HXgv091LICPAXe0drwYOBH4IXAb3TGl7f9uwJsHlns9EOA/AEdOsf530b1/i+iO99nTBLhh+7bHpa0Nr52k3puB/0F3jF4InA98KsmioXonA6fS9aergHOT/OQLjxacz5pB+wadAdxDO25JfpruC5Ef0t0S8PN0x/qf2rxx+xl07+PNdO/rCcAfJ5k4Ft+h+7s7LEmmaeOf0u3/ImAlcEGS3Yfq/AlwXNvWd4FzJtY75t8kDPWPtvxngN2BX6bra58HLm1/D49RVeuB/9WOyaA3AD8APjHNvkqSBlWVk5OTk1OPJuAK4Pj2/By6IPADumD008CPgP3a/JOAS4aW3xkoYN/2+kTghoH5dwDHDbwO8DXg8oGyy4EvDa13JfDhgddrgGOH6pwNnDFUtqi15xl0AfhB4D8PzH8S8D3grCmOyceAlUNlH+7+mfvJ688DfzhU59V0lzIH+NnWjiUD8/cAHgZOHCgr4C+H1jPOcf4W8IahOm8Bbppiv/4P8KGhsn8C1gy8Hn7/vg8sm2R9L29t2mWo/HLguhH1H/UetmVHtedv2/MFrc7ioToFHDZNneH9+DbwzhHtHN7W7w7M372V/eJA2SXAn0zzN/WYvjow78vARe35bwG3ABmYvy1dQP61cfrZwPZG9dcvDrw+Gfg3ui8T/hF4O/CsgfkT+/+OgbJt6M5kv3vo/T5ooM5+rWz+OH+Tk/UPui+87gd2HCq/FnjrFMf6lW3dzx0ouwE4bZL6xzLQ352cnJycHpk8My9J/XM53X/SoTs7+Fngyla2H10AuLLN3wd4WbuU9/4k99OdoQV49vCKkzwVeObA8lRV0Z3xHHbd0Ovb6QL5VPYBXj/Unn8eaM+zgR2ALw1s/37g+mnW+/zBZZrh1/sA7xja9seAJ9Lt8/OAHwOrBrZ9W9uvYauGXk95nNtlxHsAfzNU5z2MeB9muF/D/hz4cJJLk7wjyfOmqT/h6jHrjWrPXmMuO5Z2iff/xyN9Y8IXR2xrsB9OvFc/6YdVdUBVHf94mkMXPqF7n/cE7ht4D++l++Lm2QN1pupnE6Y8jlX1jlZ/OV3/Pxq4KckBI5abWObHdF/2zeQYTfc3OWG4f+xD9+XhuqFlX8DUffoSulstfgt+clXN3gxcYi9JGs92s90ASdKMXQ4ck2Qv4Ml0/8m+nO5S+3XA/6mqf2t1t6G7FHbUqPJ3TrGNcX7S7d+GXhfT3761Dd1/2k8ZMe/bdGfHN8R0lyJPbPtdwN+PmLduzHVM+L8j1j3Vcf7p9vz36M62bzJVdWKSc4CDgYOAE5L8XlUNXzY9bHifNsSP2+NPjmWS7R/H+kb1w+Gyn/TDqqp29fhGOVmRZFu6AfEmvtzahu7M8+Ejqq8fqDNVPxtbdQMC/j3w90mOB74C/CFdIJ6Jwb/VieO3zcDjVH+TE0b1+TvpLrsf9v3JGtLeo48Ab0zyDrovKb5aVeN+mSRJagzzktQ/XwCeALyV7rLch9MNInY6cBfdvdwTrqEbbOpfBwL+pKrq3iTfobuv+TKAdm/sz9HdxzsTP6K7BHnQNcDeVbV61AJJVtMFjyXAra3siXRn+74xxbZuassMGn59DfC8KbZ9M11A2Yfu7CZJ5tOdIZ7OdMf5viTfBp5dVTMZnftmuv0YDOLD+/UYVXUL3eXgp6YbHPG32zp+1KoMvy8zMao9N7fnE2F18J7p4Xvcp21DVX0/ye3ALwKXDsz6Rbr3enP5bWAnHrmX+xrgCODuqvreJMtM2c8GjOqvN4+qCFBVP0ryDR7bH5fQjlH7W92Xmd17PuXf5DTL7Qr8uKpuneGyH6G7reJ1dF+MjDXIpSTp0bzMXpJ6pl12fg3dgFQTo5N/ie4y7pfSBr9rPgA8Ffh4kpcm+XdJXtlGoH7yJJv4C+CtSV6T5GeB99OFs3HO1g9aA/yHdKN/T/yu+XuBfZP8dZIXJ3lOkl9O8jcD+3YG8N4kv5Rkb7rgOF34PBV4ZZLj043w/zvAa4bqnAT8RpKTkrwgyfOSHJbkT9u2vw5cDPx1kiVtoLWPAA+Mse/jHOcT6Y7r7yf52daGI9sZ18n8BbAsye+0/Tqe7j0eKcmOST6Q7vfkF7RLmAcD8L+2fXlVknmDA8XNwGuH2nMA8D8BquoHdPeYvy3J3kl+gW5Aw0F30Y3xcFC6Xw946iTb+TPg2CRHJHlukpPozgK/fyaNTTey/5+MUfXJSZ6ZZI90o7ufQve+/lVVfa7VOYfubPQFSf5jkj2TvCzJ+/PIiPZT9rMBS4b665G0s+Ptb+Jv2+NzW385FjiEbiDAQW9s6/9ZuvfhWcBpjG/Kv8kp/BPd5fgXpPuVjT2T/HySdyUZdbb+J6pqLd3f2gfpRrc/Z7hOa8ciui8vdkiyqE2z+lOQkrQlMcxLUj9dRhdwLweoqh/ShagHefT97rfT3Uf/Y+AfgBvpAsqDbRrlfXQ/IfWRtk7oAsQPZ9jGd9J9wfAN2hnbqrqObmT6BcDngK/SjbQ9eMn/sW3/zm+PN9ANKjapqvoy3eW6b6S7P/i1dOF5sM7FdL/Z/Qq6Y3Ql3Qjf3xqo9pt0o/BfTje69zl04XPKfR/nOFfVh+nuE35D2+8v0N0P/c0Rq5xY78fbfpxMd4n1C+nuiZ/Mw3T3b6+gG/n+fLovev57W9+36QZMPJnumP/VVPs1iRPpRne/ju54H1VVg2MqTIxUfhXwN8D/P7RPDwH/je6s9+3ABZNs51S6QP+ndH3gNcCvVtW1M2zvs3n0lQKTeSfd4I+rgfPo7o1/bVX914G2P0DXf2+lu/z9a3THeme6gerG7WfQvY//nu59fTfdYH8TZ9Rvohtc7n1t/pV0X94dy2NHxT+O7v39Kt0vBLymheWxjPk3OWq5ovty4VLgQ3T97Ty6W2VGjTMx7MN0x+1TVXXPJPO/Avw+3fv3lTaNc6WMJM0JE6OqSpI0qSTXAP88GGzmgnZFwe3AEVX1ydluj7YOSdbQnfEfvmphJutYQPdF0M9V1fCAjJKkOcB75iVJj5LkWXQDp32O7t+J5XS/P718Ntu1OSTZn25QwevpRvs+Gbib7my7JEnSFsMwL0ka9mO6+3f/jO52rJuAg+fI2b/t6S55/nd098pfAbysqjbGSO+SJEkbjZfZS5IkSZLUMw6AJ0mSJElSzxjmJUmSJEnqma3unvlddtmlFixYMNvNkCRJkiRpxq6++uq7q2redPW2ujC/YMECVq2aC2M0SZIkSZK2Nkn+dZx6XmYvSZIkSVLPGOYlSZIkSeoZw7wkSZIkST1jmJckSZIkqWcM85IkSZIk9YxhXpIkSZKknjHMS5IkSZLUM4Z5SZIkSZJ6xjAvSZIkSVLPGOYlSZIkSeoZw7wkSZIkST2z3Ww3QJIkSZKkDbXguM9MOX/Ne161mVqyeXlmXpIkSZKknjHMS5IkSZLUM4Z5SZIkSZJ6xjAvSZIkSVLPGOYlSZIkSeoZw7wkSZIkST0zbZhP8lNJrkzy1SQ3JnlXKz8ryTeTXNumRa08SU5NsjrJdUleMrCuZUluadOygfJ9klzfljk1SVr505KsbPVXJtl54x8CSZIkSZL6ZZwz8w8C+1fVi4BFwNIkS9q8/1FVi9p0bSs7GFjYpuXAadAFc+AE4KXAvsAJA+H8tFZ3Yrmlrfw44JKqWghc0l5LkiRJkjSnTRvmq3N/e7l9m2qKRQ4Fzm7LfRnYKcluwEHAyqpaX1X3ACvpvhjYDXhKVX2pqgo4G3j1wLpWtOcrBsolSZIkSZqzxrpnPsm2Sa4F7qIL5Fe0WSe3S+lPSfKEVrY7cNvA4mtb2VTla0eUA+xaVXcAtMdnjL1nkiRJkiRtpcYK81X1cFUtAuYD+yZ5AXA88Dzg54CnAW9r1TNqFRtQPrYky5OsSrJq3bp1M1lUkiRJkqTemdFo9lX1PeByYGlV3dEupX8Q+AjdffDQnVnfY2Cx+cDt05TPH1EOcGe7DJ/2eNck7Tq9qhZX1eJ58+bNZJckSZIkSeqdcUazn5dkp/Z8R+CVwNcGQnbo7mW/oS1yIXBkG9V+CXBvu0T+YuDAJDu3ge8OBC5u8+5LsqSt60jggoF1TYx6v2ygXJIkSZKkOWu7MersBqxIsi1d+D+vqj6d5NIk8+guk78W+L1W/yLgEGA18ABwFEBVrU/yR8BVrd5JVbW+PX8jcBawI/DZNgG8BzgvydHAt4DXbeiOSpIkSZK0tZg2zFfVdcCLR5TvP0n9Ao6ZZN6ZwJkjylcBLxhR/l3ggOnaKEmSJEnSXDKje+YlSZIkSdLsM8xLkiRJktQzhnlJkiRJknrGMC9JkiRJUs8Y5iVJkiRJ6hnDvCRJkiRJPWOYlyRJkiSpZwzzkiRJkiT1jGFekiRJkqSeMcxLkiRJktQzhnlJkiRJknrGMC9JkiRJUs8Y5iVJkiRJ6hnDvCRJkiRJPWOYlyRJkiSpZwzzkiRJkiT1jGFekiRJkqSeMcxLkiRJktQzhnlJkiRJknrGMC9JkiRJUs8Y5iVJkiRJ6hnDvCRJkiRJPWOYlyRJkiSpZwzzkiRJkiT1jGFekiRJkqSeMcxLkiRJktQzhnlJkiRJknpm2jCf5KeSXJnkq0luTPKuVr5nkiuS3JLk40l2aOVPaK9Xt/kLBtZ1fCv/epKDBsqXtrLVSY4bKB+5DUmSJEmS5rJxzsw/COxfVS8CFgFLkywB3gucUlULgXuAo1v9o4F7quo5wCmtHkn2Ag4H9gaWAh9Msm2SbYEPAAcDewFHtLpMsQ1JkiRJkuasacN8de5vL7dvUwH7A59o5SuAV7fnh7bXtPkHJEkrP7eqHqyqbwKrgX3btLqqbq2qHwHnAoe2ZSbbhiRJkiRJc9ZY98y3M+jXAncBK4FvAN+rqodalbXA7u357sBtAG3+vcDTB8uHlpms/OlTbEOSJEmSpDlrrDBfVQ9X1SJgPt2Z9OePqtYeM8m8jVX+GEmWJ1mVZNW6detGVZEkSZIkaasxo9Hsq+p7wOXAEmCnJNu1WfOB29vztcAeAG3+U4H1g+VDy0xWfvcU2xhu1+lVtbiqFs+bN28muyRJkiRJUu+MM5r9vCQ7tec7Aq8EbgYuAw5r1ZYBF7TnF7bXtPmXVlW18sPbaPd7AguBK4GrgIVt5Pod6AbJu7AtM9k2JEmSJEmas7abvgq7ASvaqPPbAOdV1aeT3AScm+TdwFeAM1r9M4CPJllNd0b+cICqujHJecBNwEPAMVX1MECSNwEXA9sCZ1bVjW1db5tkG5IkSZIkzVnThvmqug548YjyW+nunx8u/yHwuknWdTJw8ojyi4CLxt2GJEmSJElz2YzumZckSZIkSbPPMC9JkiRJUs8Y5iVJkiRJ6hnDvCRJkiRJPWOYlyRJkiSpZwzzkiRJkiT1jGFekiRJkqSeMcxLkiRJktQzhnlJkiRJknrGMC9JkiRJUs8Y5iVJkiRJ6hnDvCRJkiRJPWOYlyRJkiSpZwzzkiRJkiT1jGFekiRJkqSeMcxLkiRJktQzhnlJkiRJknrGMC9JkiRJUs8Y5iVJkiRJ6hnDvCRJkiRJPWOYlyRJkiSpZwzzkiRJkiT1jGFekiRJkqSeMcxLkiRJktQzhnlJkiRJknrGMC9JkiRJUs9MG+aT7JHksiQ3J7kxyZtb+YlJvp3k2jYdMrDM8UlWJ/l6koMGype2stVJjhso3zPJFUluSfLxJDu08ie016vb/AUbc+clSZIkSeqjcc7MPwT8QVU9H1gCHJNkrzbvlKpa1KaLANq8w4G9gaXAB5Nsm2Rb4APAwcBewBED63lvW9dC4B7g6FZ+NHBPVT0HOKXVkyRJkiRpTps2zFfVHVV1TXt+H3AzsPsUixwKnFtVD1bVN4HVwL5tWl1Vt1bVj4BzgUOTBNgf+ERbfgXw6oF1rWjPPwEc0OpLkiRJkjRnzeie+XaZ+4uBK1rRm5Jcl+TMJDu3st2B2wYWW9vKJit/OvC9qnpoqPxR62rz7231JUmSJEmas8YO80meBHwSeEtVfR84DXg2sAi4A3j/RNURi9cGlE+1ruG2LU+yKsmqdevWTbkfkiRJkiT13VhhPsn2dEH+nKr6FEBV3VlVD1fVj4EP0V1GD92Z9T0GFp8P3D5F+d3ATkm2Gyp/1Lra/KcC64fbV1WnV9Xiqlo8b968cXZJkiRJkqTeGmc0+wBnADdX1Z8PlO82UO01wA3t+YXA4W0k+j2BhcCVwFXAwjZy/Q50g+RdWFUFXAYc1pZfBlwwsK5l7flhwKWtviRJkiRJc9Z201dhP+ANwPVJrm1lb6cbjX4R3WXva4DfBaiqG5OcB9xENxL+MVX1MECSNwEXA9sCZ1bVjW19bwPOTfJu4Ct0Xx7QHj+aZDXdGfnDH8e+SpIkSZK0VZg2zFfVFxl97/pFUyxzMnDyiPKLRi1XVbfyyGX6g+U/BF43XRslSZIkSZpLZjSavSRJkiRJmn2GeUmSJEmSesYwL0mSJElSzxjmJUmSJEnqGcO8JEmSJEk9Y5iXJEmSJKlnDPOSJEmSJPWMYV6SJEmSpJ4xzEuSJEmS1DOGeUmSJEmSesYwL0mSJElSzxjmJUmSJEnqGcO8JEmSJEk9Y5iXJEmSJKlnDPOSJEmSJPWMYV6SJEmSpJ4xzEuSJEmS1DOGeUmSJEmSesYwL0mSJElSzxjmJUmSJEnqGcO8JEmSJEk9Y5iXJEmSJKlnDPOSJEmSJPWMYV6SJEmSpJ4xzEuSJEmS1DOGeUmSJEmSesYwL0mSJElSz0wb5pPskeSyJDcnuTHJm1v505KsTHJLe9y5lSfJqUlWJ7kuyUsG1rWs1b8lybKB8n2SXN+WOTVJptqGJEmSJElz2Thn5h8C/qCqng8sAY5JshdwHHBJVS0ELmmvAQ4GFrZpOXAadMEcOAF4KbAvcMJAOD+t1Z1Ybmkrn2wbkiRJkiTNWdOG+aq6o6quac/vA24GdgcOBVa0aiuAV7fnhwJnV+fLwE5JdgMOAlZW1fqqugdYCSxt855SVV+qqgLOHlrXqG1IkiRJkjRnzeie+SQLgBcDVwC7VtUd0AV+4Bmt2u7AbQOLrW1lU5WvHVHOFNsYbtfyJKuSrFq3bt1MdkmSJEmSpN4ZO8wneRLwSeAtVfX9qaqOKKsNKB9bVZ1eVYuravG8efNmsqgkSZIkSb0zVphPsj1dkD+nqj7Viu9sl8jTHu9q5WuBPQYWnw/cPk35/BHlU21DkiRJkqQ5a5zR7AOcAdxcVX8+MOtCYGJE+mXABQPlR7ZR7ZcA97ZL5C8GDkyycxv47kDg4jbvviRL2raOHFrXqG1IkiRJkjRnbTdGnf2ANwDXJ7m2lb0deA9wXpKjgW8Br2vzLgIOAVYDDwBHAVTV+iR/BFzV6p1UVevb8zcCZwE7Ap9tE1NsQ5IkSZKkOWvaMF9VX2T0fe0AB4yoX8Axk6zrTODMEeWrgBeMKP/uqG1IkiRJkjSXzWg0e0mSJEmSNPsM85IkSZIk9YxhXpIkSZKknjHMS5IkSZLUM4Z5SZIkSZJ6xjAvSZIkSVLPGOYlSZIkSeoZw7wkSZIkST1jmJckSZIkqWcM85IkSZIk9YxhXpIkSZKknjHMS5IkSZLUM4Z5SZIkSZJ6xjAvSZIkSVLPGOYlSZIkSeoZw7wkSZIkST1jmJckSZIkqWcM85IkSZIk9YxhXpIkSZKknjHMS5IkSZLUM4Z5SZIkSZJ6xjAvSZIkSVLPGOYlSZIkSeoZw7wkSZIkST1jmJckSZIkqWcM85IkSZIk9cy0YT7JmUnuSnLDQNmJSb6d5No2HTIw7/gkq5N8PclBA+VLW9nqJMcNlO+Z5IoktyT5eJIdWvkT2uvVbf6CjbXTkiRJkiT12Thn5s8Clo4oP6WqFrXpIoAkewGHA3u3ZT6YZNsk2wIfAA4G9gKOaHUB3tvWtRC4Bzi6lR8N3FNVzwFOafUkSZIkSZrzpg3zVfV5YP2Y6zsUOLeqHqyqbwKrgX3btLqqbq2qHwHnAocmCbA/8Im2/Arg1QPrWtGefwI4oNWXJEmSJGlOezz3zL8pyXXtMvydW9nuwG0Ddda2ssnKnw58r6oeGip/1Lra/HtbfUmSJEmS5rQNDfOnAc8GFgF3AO9v5aPOnNcGlE+1rsdIsjzJqiSr1q1bN1W7JUmSJEnqvQ0K81V1Z1U9XFU/Bj5Edxk9dGfW9xioOh+4fYryu4Gdkmw3VP6odbX5T2WSy/2r6vSqWlxVi+fNm7chuyRJkiRJUm9sUJhPstvAy9cAEyPdXwgc3kai3xNYCFwJXAUsbCPX70A3SN6FVVXAZcBhbfllwAUD61rWnh8GXNrqS5IkSZI0p203XYUkfwe8HNglyVrgBODlSRbRXfa+BvhdgKq6Mcl5wE3AQ8AxVfVwW8+bgIuBbYEzq+rGtom3AecmeTfwFeCMVn4G8NEkq+nOyB/+uPdWkiRJkqStwLRhvqqOGFF8xoiyifonAyePKL8IuGhE+a08cpn+YPkPgddN1z5JkiRJkuaaxzOavSRJkiRJmgWGeUmSJEmSesYwL0mSJElSzxjmJUmSJEnqGcO8JEmSJEk9Y5iXJEmSJKlnDPOSJEmSJPWMYV6SJEmSpJ4xzEuSJEmS1DOGeUmSJEmSesYwL0mSJElSzxjmJUmSJEnqGcO8JEmSJEk9Y5iXJEmSJKlnDPOSJEmSJPWMYV6SJEmSpJ4xzEuSJEmS1DOGeUmSJEmSesYwL0mSJElSzxjmJUmSJEnqGcO8JEmSJEk9Y5iXJEmSJKlnDPOSJEmSJPWMYV6SJEmSpJ4xzEuSJEmS1DOGeUmSJEmSembaMJ/kzCR3JblhoOxpSVYmuaU97tzKk+TUJKuTXJfkJQPLLGv1b0mybKB8nyTXt2VOTZKptiFJkiRJ0lw3zpn5s4ClQ2XHAZdU1ULgkvYa4GBgYZuWA6dBF8yBE4CXAvsCJwyE89Na3Ynllk6zDUmSJEmS5rRpw3xVfR5YP1R8KLCiPV8BvHqg/OzqfBnYKcluwEHAyqpaX1X3ACuBpW3eU6rqS1VVwNlD6xq1DUmSJEmS5rTtNnC5XavqDoCquiPJM1r57sBtA/XWtrKpyteOKJ9qG5IkSZKkLdyC4z4zbZ0173nVZmjJ1mlDw/xkMqKsNqB8ZhtNltNdqs/P/MzPzHRxSZIkSdIsmC7wG/Ynt6Gj2d/ZLpGnPd7VytcCewzUmw/cPk35/BHlU23jMarq9KpaXFWL582bt4G7JEmSJElSP2xomL8QmBiRfhlwwUD5kW1U+yXAve1S+YuBA5Ps3Aa+OxC4uM27L8mSNor9kUPrGrUNSZIkSZLmtGkvs0/yd8DLgV2SrKUblf49wHlJjga+BbyuVb8IOARYDTwAHAVQVeuT/BFwVat3UlVNDKr3RroR83cEPtsmptiGJEmSJElz2rRhvqqOmGTWASPqFnDMJOs5EzhzRPkq4AUjyr/RV7wnAAAJnklEQVQ7ahuSJEmSJM11G3qZvSRJkiRJmiWGeUmSJEmSesYwL0mSJElSz2zs35mXJEmSJGmjmO536Ocyz8xLkiRJktQzhnlJkiRJknrGMC9JkiRJUs8Y5iVJkiRJ6hnDvCRJkiRJPWOYlyRJkiSpZwzzkiRJkiT1jGFekiRJkqSeMcxLkiRJktQzhnlJkiRJknrGMC9JkiRJUs8Y5iVJkiRJ6hnDvCRJkiRJPWOYlyRJkiSpZwzzkiRJkiT1jGFekiRJkqSeMcxLkiRJktQzhnlJkiRJknrGMC9JkiRJUs8Y5iVJkiRJ6hnDvCRJkiRJPWOYlyRJkiSpZwzzkiRJkiT1zOMK80nWJLk+ybVJVrWypyVZmeSW9rhzK0+SU5OsTnJdkpcMrGdZq39LkmUD5fu09a9uy+bxtFeSJEmSpK3Bxjgz/4qqWlRVi9vr44BLqmohcEl7DXAwsLBNy4HToAv/wAnAS4F9gRMmvgBodZYPLLd0I7RXkiRJkqRe2xSX2R8KrGjPVwCvHig/uzpfBnZKshtwELCyqtZX1T3ASmBpm/eUqvpSVRVw9sC6JEmSJEmasx5vmC/gH5NcnWR5K9u1qu4AaI/PaOW7A7cNLLu2lU1VvnZE+WMkWZ5kVZJV69ate5y7JEmSJEnSlm27x7n8flV1e5JnACuTfG2KuqPud68NKH9sYdXpwOkAixcvHllHkiRJkrRxLDjuM7PdhDnvcZ2Zr6rb2+NdwPl097zf2S6Rpz3e1aqvBfYYWHw+cPs05fNHlEuSJEmSNKdtcJhP8sQkT554DhwI3ABcCEyMSL8MuKA9vxA4so1qvwS4t12GfzFwYJKd28B3BwIXt3n3JVnSRrE/cmBdkiRJkiTNWY/nMvtdgfPbr8VtB3ysqv4hyVXAeUmOBr4FvK7Vvwg4BFgNPAAcBVBV65P8EXBVq3dSVa1vz98InAXsCHy2TZIkSZIkzWkbHOar6lbgRSPKvwscMKK8gGMmWdeZwJkjylcBL9jQNkqSJEmStDXaFD9NJ0mSJEmSNiHDvCRJkiRJPWOYlyRJkiSpZwzzkiRJkiT1zOMZzV6SJEmStBVacNxnZrsJmoZn5iVJkiRJ6hnDvCRJkiRJPWOYlyRJkiSpZ7xnXpIkSZLmEO+H3zp4Zl6SJEmSpJ7xzLwkSZIkbSU86z53GOYlSZIkqScM65rgZfaSJEmSJPWMZ+YlSZIkaQvgWXfNhGFekiRJkh4ng7g2N8O8JEmSJE3BoK4tkWFekiRJUi8ZsjWXGeYlSZIkbXQGbWnTcjR7SZIkSZJ6xjAvSZIkSVLPGOYlSZIkSeoZ75mXJEmSNCPeDy/NPs/MS5IkSZLUM4Z5SZIkSZJ6xjAvSZIkSVLPGOYlSZIkSeqZLT7MJ1ma5OtJVic5brbbI0mSJEnSbNuiw3ySbYEPAAcDewFHJNlrdlslSZIkSdLs2qLDPLAvsLqqbq2qHwHnAofOcpskSZIkSZpVW/rvzO8O3Dbwei3w0llqiyT12nS/CbzmPa/aTC3ZPDbGbyBvbcdEksbhb8hL/bClh/mMKKvHVEqWA8vby/uTfH2TtmrD7QLcPduN0BbPfqJxbdS+kvdurDVtPbaSY+JnisZhP9G47CsaxxbVT3r47/mzxqm0pYf5tcAeA6/nA7cPV6qq04HTN1ejNlSSVVW1eLbboS2b/UTjsq9oHPYTjcN+onHZVzQO+8nmsaXfM38VsDDJnkl2AA4HLpzlNkmSJEmSNKu26DPzVfVQkjcBFwPbAmdW1Y2z3CxJkiRJkmbVFh3mAarqIuCi2W7HRrLF3wqgLYL9ROOyr2gc9hONw36icdlXNA77yWaQqseMJydJkiRJkrZgW/o985IkSZIkaYhhfiNLskeSy5LcnOTGJG8eUSdJTk2yOsl1SV4yG23V7Bmzn7w8yb1Jrm3TO2ejrZo9SX4qyZVJvtr6ybtG1HlCko+3z5MrkizY/C3VbBuzr/xmknUDnym/PRtt1exLsm2SryT59Ih5fqYImLaf+HkiAJKsSXJ96werRsw392xCW/w98z30EPAHVXVNkicDVydZWVU3DdQ5GFjYppcCp7VHzR3j9BOAL1TVL89C+7RleBDYv6ruT7I98MUkn62qLw/UORq4p6qek+Rw4L3Ar89GYzWrxukrAB+vqjfNQvu0ZXkzcDPwlBHz/EzRhKn6Cfh5oke8oqom+015c88m5Jn5jayq7qiqa9rz++g+BHcfqnYocHZ1vgzslGS3zdxUzaIx+4nmuPYZcX97uX2bhgc6ORRY0Z5/AjggSTZTE7WFGLOvSCSZD7wK+PAkVfxM0Tj9RBqXuWcTMsxvQu3StBcDVwzN2h24beD1Wgxyc9YU/QTg59tls59NsvdmbZi2CO0yx2uBu4CVVTXp50lVPQTcCzx987ZSW4Ix+grAr7bLHD+RZI/N3ERtGf4n8Fbgx5PM9zNFMH0/AT9P1CngH5NcnWT5iPnmnk3IML+JJHkS8EngLVX1/eHZIxbxDMocNE0/uQZ4VlW9CPhL4H9t7vZp9lXVw1W1CJgP7JvkBUNV/DwRMFZf+d/Agqr698A/8cjZV80RSX4ZuKuqrp6q2ogyP1PmkDH7iZ8nmrBfVb2E7nL6Y5K8bGi+nymbkGF+E2j3K34SOKeqPjWiylpg8BvM+cDtm6Nt2nJM10+q6vsTl81W1UXA9kl22czN1Baiqr4HXA4sHZr1k8+TJNsBTwXWb9bGaYsyWV+pqu9W1YPt5YeAfTZz0zT79gN+Jcka4Fxg/yR/O1THzxRN20/8PNGEqrq9Pd4FnA/sO1TF3LMJGeY3snZf2RnAzVX155NUuxA4so3uuAS4t6ru2GyN1Kwbp58keebEfYpJ9qX7e/3u5mulZluSeUl2as93BF4JfG2o2oXAsvb8MODSqvIb7zlmnL4ydI/ir9CN1aE5pKqOr6r5VbUAOJzu8+L1Q9X8TJnjxuknfp4IIMkT20DOJHkicCBww1A1c88m5Gj2G99+wBuA69u9iwBvB34GoKr+GrgIOARYDTwAHDUL7dTsGqefHAa8MclDwA+Aw/0P1ZyzG7AiybZ0X+acV1WfTnISsKqqLqT7UuijSVbTnT07fPaaq1k0Tl/5b0l+he7XNNYDvzlrrdUWxc8UjcPPE42wK3B+O/e0HfCxqvqHJL8H5p7NIWYDSZIkSZL6xcvsJUmSJEnqGcO8JEmSJEk9Y5iXJEmSJKlnDPOSJEmSJPWMYV6SJEmSpJ4xzEuSJEmS1DOGeUmSJEmSesYwL0mSJElSz/w/sVr6QyMGi/YAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1224x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(Gold.dw, bins=100)\n",
    "plt.title('weighted degree distribution: DeepSphere V1', fontsize=14)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5,1,'weighted degree distribution: DeepSphere V2\\nneighbors=50')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA/MAAAFQCAYAAADz6y7KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAAIABJREFUeJzt3XvYXFV59/HvDyIeahGUiBSo4dVYBa0RIqbVtgoqQdtGLbRQlVSxqMXWE1XQt4JUWg8oLa1iqSDBooinQhWLKQcPfRWIiBxFUkQIIARBhCoocL9/7PXoMMxzCkme7Dzfz3XNNbPvvfbaa+/ZTzL3rLXXpKqQJEmSJEn9sclMN0CSJEmSJE2PybwkSZIkST1jMi9JkiRJUs+YzEuSJEmS1DMm85IkSZIk9YzJvCRJkiRJPWMyL0mzXJLDklwyzW3OSfLP66pNI/b3+SQnTHObvZJsdL+/muSgJFcPLE/7/ZvGvq5OctDAciXZax3ta50dx2yV5NntPdtqptsiSVr7TOYlSUcCv7e2Kx1OBLXOTPn9W4Pk7unAh9a4ZaPbMK+1YeHQqnVyHU6xTVe3NlWSO5Ncm+RzSf5gJtrT2vTUJKcm+UFr0zVJPpPksTPVpgcqyUVJPjLOuhe08/+EduyfaO/DT5NckeSvk/i5VZIG+I+iJM1yVXVHVf1wptvRV0k2m8n9r4v3b+yYqmp1Vf1kbdY9ng3gOjwc2AZ4ArAPcDXwuST/tL4bkmQucCZwB/BC4InAy4H/ATZf3+0Z9gCu+eOAP0nyKyPWvRL4alV9F9gFWE13zDsBhwLvAA5ew/1K0kbJZF6SeiTJnkluTzKnLc9vvVnHDJQ5IsnygeUdk3yhbXdT6/F6zMD6+wxvTjInyVFJbm2Po5Ick+ScoeZskuTvktzc6j1yrOeslX0s8L6xHs+B+n87yZeT/CTJda3uzQfWPyzJCUnuSHJjkrdN8dzsl+T7rd7PA1uPKPMHSb7Zejq/187VZgPrt05yWusN/H6SVyS5JMlhA2UqyYFJPpvkf4G/m8p5bmVekeSytv/vJnnjZL2NSd7SemfvSHIi8PCh9cPv31OSnJnkx60t307ynCTzgLNbsdXtOE5o25zT3ocjk6wG/rvFR42ueEw7zp+0c/SygX2P7HXPfYfnf689n9/i54xzHJsk+Zt0vbN3Jbk4yZIR+/qjJMtbey5L8ryJzucEbq+qH1TVNVX131X1RuAvgNclec7AfrdNcvLA38cXkswfOt7JrrOr2/H+W3tffzB0np8JbAm8oqq+WVVXV9WXq+otVXXx0PH/aZKvtX19J8nzRxzbU5Oc287RiiQ7D7V3sr/J8a6PRyQ5tl3vt7c6hkdcDPoY8CDgj4f2Pxf4Q+AjAFV1fFX9VVWdU1VXVdXJwDHAH01QtyTNOibzktQvXwUeAox9YH42cDPwnIEyzwbOAUiyDfAV4BJgV+C5dMngaRMkkQcBfwa8ClhE93/Fn44o91LgbuC3gdcBbwD+pK17CbCKX/Z2btPa8xTgS8BpwFNbuQXA8QP1Hgk8j+6D++7A04DfHaettHqfAZwAHNvq+4+278EyewAnAf9M19v3SmAvWjLeLKP7EmI3YAnwsrY87FDgdOApwAencp6T/Hnb1zuAJwFvBt5KlzCOd1x/DLyr7W9n4ArgTROdC+DjwA2tHU8DDgPuBK7ll8nQTnTvyesHtnsZEOB3gP0mqP+ddO/fArrzfeIkCdywXdvz4taGl4xT7vXAX9Odo6cAnwM+m2TBULkjgKPprqfzgZOT/OILj5Y4nzCN9g06DriVdt6SPIzuC5E76W4J+C26c/1fbd1UrzPo3sfL6d7XQ4G/SzJ2Ln5A93e3V5JM0sb30h3/AmA5cGqSbYfK/D1dr/bOwA+Bk8bqneLfJAxdH237LwDbAr9Pd619BTir/T3cT1XdAvx7OyeDXg78FPj0BMe5Od17IUkaU1U+fPjw4aNHD+Bc4JD2+iS6ROCndInRw4CfAc9s6w8HzhzafkuggF3b8mHAJQPrbwAOHlgO8B3gnIHYOcDXh+pdDnxkYPlq4KChMicCxw3FFrT2PJouAb4LeOnA+ocDPwJOmOCcfBxYPhT7SPff3C+WvwL8zVCZF9ENZQ7wG60diwbWbw/cAxw2ECvgn4bqmcp5vgZ4+VCZNwCXTXBc/w/416HYfwFXDywPv38/BpaOU9+zW5u2GoqfA1w0ovx93sO27aj2/Ft7Pa+VWThUpoC9JikzfBzXAe8Y0c7hfb16YP22LfasgdiZwN9P8jd1v2t1YN03gNPb61cCVwIZWL8pXYL8x1O5zgb2N+p6/drA8hHAz+kS2C8BbwMeO7B+7PjfPhDbBPgu8K6h93uPgTLPbLHtpvI3Od71QfeF1x3AQ4fiFwJvmeBcP7fV/YSB2CXAMRNsszPdFyh/NNH76MOHDx+z7WHPvCT1zzl0H9Kh6x38InBeiz2TLgE4r63fBfjdNpT3jiR30PXQAjxuuOIkjwAeM7A9VVV0PZ7DLhpavp4uIZ/ILsDLhtrz3wPteRywGfD1gf3fAVw8Sb1PGtymGV7eBXj70L4/DvwK3TE/EbgXWDGw72vbcQ1bMbQ84Xluw4i3B/5lqMy7GfE+TPO4hn0A+EiSs5K8PckTJyk/5ptTLDeqPTtOcdspaUO8f41fXhtjvjZiX4PX4dh79YvrsKp2r6pDHkhz6JJP6N7nHYDbB97D2+i+uHncQJmJrrMxE57Hqnp7K38A3fW/P3BZkt1HbDe2zb10X/ZN5xxN9jc5Zvj62IXuy8PVQ9s+mYmv6TPpbrV4JfxiVM1OtCH2w5L8Bt0IgH+oqs9MUK8kzTpzZroBkqRpOwc4MMmOwK/Sfcg+h26o/Wrg/1XVz1vZTeg+CI+aVf7GCfYxlZ90+/nQcjH57Vub0H1oP2rEuuvoesfXxGRDkcf2/U7gUyPWrZ5iHWP+d0TdE53nh7XXr6HrbV9nquqwJCcBewJ7AIcmeU1VDQ+bHjZ8TGvi3vb8i3OZ5EEPoL5R1+Fw7BfXYVVVGz2+VjorkmxKNyHe2Jdbm9D1PO8zovgtA2Umus6mrLoJAT8FfCrJIcC3gL+hS4inY/Bvdez8bTLwPNHf5JhR1/yNdMPuh/14vIa09+ijwGuTvJ3uS4pvV9X9vkxqX0SdDZxcVU5+J0lDTOYlqX++CjwYeAvdsNx70k0idixwE9293GMuoJts6vsDCf64quq2JD+gu6/5bIB2b+zT6e7jnY6f0Q1BHnQBsFNVrRy1QZKVdInHIuCqFvsVut6+/5lgX5e1bQYNL18APHGCfV9Ol6DsQte7SZLt6HqIJzPZeb49yXXA46rqxCnUN+ZyuuMYTMSHj+t+qupKuuHgR6ebHPFVrY6ftSLD78t0jGrP5e31WLI6eM/08D3uk7ahqn6c5HrgWcBZA6ueRfdery+vArbgl/dyXwDsC9xcVT8aZ5sJr7MBo67Xy0cVBKiqnyX5H+5/PS6inaP2t7orE997Pqq94/5NTrLd1sC9VXXVNLf9KN1tFXvTfTFyv0ku25eVZwGnVDcZoSRpiMPsJaln2rDzC+gmpBqbnfzrdMO4n0Gb/K75IPAI4JNJnpHk/yR5bpuB+lfH2cU/Am9J8uI2xPX9dMnZVHrrB10N/E662b/Hftf8PcCuST6c5GlJHp/k95P8y8CxHQe8J8nzkuxElzhOlnweDTw3ySHpZvj/c+DFQ2UOB/40yeFJnpzkiUn2SvLetu8rgDOADydZ1CZa+yjwkykc+1TO82F05/WNSX6jtWG/1uM6nn8Elib583Zch9C9xyMleWiSD6b7Pfl5bQjzYAL8/XYsL0wyd3CiuGl4yVB7dgf+AaCqfkp3j/lbk+yU5LfpJjQcdBPdHA97pPv1gEeMs5/3AQcl2Tfdb48fTtcL/P7pNDbdzP5/P4Wiv5rkMUm2Tze7+1F07+s/V9WXW5mT6HqjT03ye0l2SPK7Sd6fX85oP+F1NmDR0PW6H613vP1N/Ft7fkK7Xg4CXkA3EeCg17b6f4PufXgs3czvUzXh3+QE/otuOP6p6X5lY4ckv5XknUlG9db/QlWtovtb+xDd7PYnDa5vf/dn0/1b9nftfXlMhn4dQpJmO5N5Seqns+kS3HMAqupOuiTqLu57v/v1dPfR3wv8J3ApXYJyV3uMciTdT0h9tNUJXQJx5zTb+A66Lxj+h9ZjW1UX0c1MPw/4MvBtupm2B4f8H9SO73Pt+RK6ScXGVVXfoBuu+1q6+4NfQpc8D5Y5g+43u59Dd47Oo5vh+5qBYn9GNwv/OXSze59El3xOeOxTOc9V9RG6+4Rf3o77q3T3Q39vRJVj9X6yHccRdEOsn0J3T/x47qG7f3sZ3cz3n6P7oudNrb7r6CZMPILunP/zRMc1jsPoZne/iO58v6KqBudUGJup/HzgX4D/O3RMdwN/RdfrfT1w6jj7OZouoX8v3TXwYroJ0C6cZnsfx31HCoznHXSTP64ETqG7N/4lVfWXA23/Cd31exXd8Pfv0J3rLWkzrU/xOoPuffxNuvf1XXST/Y31qF9GN7nckW39eXRf3h3E/WfFP5ju/f023S8EvLgly1Myxb/JUdsV3ZcLZwH/Sne9nUJ3q8yoeSaGfYTuvH22qoZnqd+b7p7+P6F7TwYfkqRmbFZVSZLGleQC4L8HE5vZoI0ouB7Y18m3tLYkuZqux3941MJ06phH90XQ06tqeEJGSdIs4D3zkqT7SPJYuonTvkz3/8QBdL8/fcBMtmt9SLIb3aSCF9P1DB4B3EzX2y5JkrTBMJmXJA27l+7+3ffR3Y51GbDnLOn9exDdkOf/Q3ev/LnA71bV2pjpXZIkaa1xmL0kSZIkST3jBHiSJEmSJPWMybwkST3SfnKukiycxjZ/luSOScocluSSB95CSZK0PpjMS5LUL9fS/dTadH+ibaPRvswYfrxmqMxTknw5yU+TXJfkHUkyU22WJGltcwI8SZJ6pKruAX4w0+2YqiQPqqqfr4Oq/xz4/MDybQP73BxYDnwFeDrdb5+fAPwv8P510BZJktY7e+YlSVqPkpyT5ENJ/i7JzUluSnJkkk3a+s2SvCfJqiT/m+T8JHsMbH+/YfZJXpjkiiR3JvlKkn1amXlD+949ySWt3rOT7DCifa9Kck3r0f73JFsNrNskyd8kuTbJXUkuTrJkRNv2TXJWkp8Cr07yiCQfa8d6Z5KrkrzhAZ7KH1XVDwYePx1Y91LgYcDSqrqkqj4DvAd4k73zkqSNhcm8JEnr30uBu4HfBl4HvAH4k7buo8DvAX8KPAVYBvxHkqeOqijJrwOfBb4APBU4GnjviKIPBg4BXgn8FrAF8OGhMvOAlwFLgOcC84HjB9a/Hvhr4K2tbZ8DPptkwVA9fw98CNgR+He6n/t7CvD7wBNbG64bOIYvJrljoseI4/nH9mXI+UleM/ZlSPNbwFeHEvwzgF9rxyhJUu85zF6SpPXvsqp6R3v93SR/Duye5DxgX2BeVV3T1v9zkucCrwb+YkRdrwWuAt5c3e/NXpHkCcARQ+XmAAdW1RUASY4EPppkk6q6t5V5KLDf2L6TvBr4apL5VXUlcBBwZFV9vJV/R5LfbfGXDezrn6rq02MLSR4LfKuqzmuhq4fa9qq276l6B3A2cAewO93Q+a3ovjQAeAywamibGwfWfW8a+5IkaYNkMi9J0vp30dDy9cCjgZ2BAJcNjQZ/MHDWOHU9ETi/JfJjzh1R7q6xRH5gnw+i66G/pcWuG/gSYayee4EnJbmRrmf7v4fq/RrwgqHYiqHlY4BPJ9mZ7l72/6iqL4+trKrrmIaq+tuBxQuTbAq8nV8m8wB1363IOHFJknrJZF6SpPVveEK4orv1bZP2+ukjyvyU0cLUEtS7R+wTpn/L3ah9Dcf+9z4rq77Yeuf3pOtJ/0KST1XVK6AbZg/8zoQ7rXr4BKvPBTZPsnVV3Ug3QeBjhso8uj3fiCRJGwGTeUmSNhzfokvOH1NVZ09xm8vp7nEftOsa7n/bJNtX1bUD9WwCXF5VP05yPfAs7jtK4FnAZZNVXFU3Ax8DPtaS908keU1V3cX0h9kPWwDcCfyoLX8deE+Sh1TVnS32PLrRCFc/gP1IkrTBMJmXJGkDUVXfTXIScEKSNwMXAI8Eng1cVVWfHbHZh+lmaT8S+FdgJ7r762H6Q8p/CixL8ia65PrDwBfa/fIA7wMOT3Il8E26++R/B9hlokqTHN6O5VK6zx4vacdzVzvuKQ+zT/IHdL3uX2/tfQ5wOHDsWH3Ax4FD6c7ju4AnAAcD7xy6HUGSpN4ymZckacPyCrr7v98LbEd3P/t5dBO+3U9VfT/JHwEfoJsZ/3zgnXSz0N85apsJXA2cDPwH3YRyX6LrNR9zNPCrrW1bA1cAf1RVF05S7110E/Lt0Nr0DeAPptm2MT+nmwjwA3SjBq6imxDvg2MFquq2JM9rsRXArXST5H1gDfcpSdIGJ35BLUnSxiXJ6+l6q7ccmKlekiRtROyZlySp55IcSNcjvxpYBPwNcIKJvCRJGy+TeUmS+u/xwNuAR9H9vvqH6XrmJUnSRsph9pIkSZIk9cx0f1tWkiRJkiTNMJN5SZIkSZJ6ZqO7Z36rrbaqefPmzXQzJEmSJEmatm9+85s3V9XcycptdMn8vHnzWLFixUw3Q5IkSZKkaUvy/amUc5i9JEmSJEk9YzIvSZIkSVLPmMxLkiRJktQzJvOSJEmSJPWMybwkSZIkST1jMi9JkiRJUs+YzEuSJEmS1DMm85IkSZIk9YzJvCRJkiRJPWMyL0mSJElSz5jMS5IkSZLUM3NmugGSJEmSNBvMO/gLE66/+t0vXE8t0cZgyj3zSTZN8q0kn2/LOyQ5N8mVST6ZZLMWf3BbXtnWzxuo45AWvyLJHgPxxS22MsnBA/GR+5AkSZIkaTabzjD71wOXDyy/BziqquYDtwL7t/j+wK1V9XjgqFaOJDsC+wA7AYuBD7UvCDYFPgjsCewI7NvKTrQPSZIkSZJmrSkl80m2A14IfKQtB9gN+HQrsgx4UXu9pC3T1u/eyi8BTq6qu6rqe8BKYNf2WFlVV1XVz4CTgSWT7EOSJEmSpFlrqj3z/wC8Bbi3LT8K+FFV3d2WVwHbttfbAtcCtPW3tfK/iA9tM158on3cR5IDkqxIsmL16tVTPCRJkiRJkvpp0mQ+ye8DN1XVNwfDI4rWJOvWVvz+wapjq2phVS2cO3fuqCKSJEmSJG00pjKb/TOBP0zyAuAhwOZ0PfVbJJnTes63A65v5VcB2wOrkswBHgHcMhAfM7jNqPjNE+xDkiRJkqRZa9JkvqoOAQ4BSPJs4KCqemmSTwF70d3jvhQ4tW1yWlv+elt/VlVVktOAjyf5APBrwHzgPLoe+PlJdgCuo5sk70/bNmePsw9JkiRJ2qhM9tN14M/X6ZemM5v9sLcCb0qyku7+9uNa/DjgUS3+JuBggKq6FDgFuAz4T+DAqrqn9bq/DjiDbrb8U1rZifYhSZIkSdKsNZVh9r9QVecA57TXV9HNRD9c5k5g73G2PwI4YkT8dOD0EfGR+5AkSZIkaTZ7ID3zkiRJkiRpBpjMS5IkSZLUMybzkiRJkiT1jMm8JEmSJEk9YzIvSZIkSVLPmMxLkiRJktQzJvOSJEmSJPWMybwkSZIkST1jMi9JkiRJUs+YzEuSJEmS1DMm85IkSZIk9YzJvCRJkiRJPWMyL0mSJElSz5jMS5IkSZLUMybzkiRJkiT1jMm8JEmSJEk9YzIvSZIkSVLPmMxLkiRJktQzJvOSJEmSJPWMybwkSZIkST1jMi9JkiRJUs+YzEuSJEmS1DMm85IkSZIk9cykyXyShyQ5L8m3k1ya5J0tfkKS7yW5sD0WtHiSHJ1kZZKLkuw8UNfSJFe2x9KB+C5JLm7bHJ0kLf7IJMtb+eVJtlz7p0CSJEmSpH6ZSs/8XcBuVfVUYAGwOMmitu6vq2pBe1zYYnsC89vjAOAY6BJz4FDgGcCuwKEDyfkxrezYdotb/GDgzKqaD5zZliVJkiRJmtUmTearc0dbfFB71ASbLAFObNt9A9giyTbAHsDyqrqlqm4FltN9MbANsHlVfb2qCjgReNFAXcva62UDcUmSJEmSZq0p3TOfZNMkFwI30SXk57ZVR7Sh9EcleXCLbQtcO7D5qhabKL5qRBxg66q6AaA9P3qc9h2QZEWSFatXr57KIUmSJEmS1FtTSuar6p6qWgBsB+ya5MnAIcATgacDjwTe2opnVBVrEJ+yqjq2qhZW1cK5c+dOZ1NJkiRJknpnWrPZV9WPgHOAxVV1QxtKfxfwUbr74KHrWd9+YLPtgOsniW83Ig5wYxuGT3u+aTrtlSRJkiRpYzSV2eznJtmivX4o8FzgOwNJdujuZb+kbXIasF+b1X4RcFsbIn8G8PwkW7aJ754PnNHW3Z5kUatrP+DUgbrGZr1fOhCXJEmSJGnWmjOFMtsAy5JsSpf8n1JVn09yVpK5dMPkLwRe08qfDrwAWAn8BHgFQFXdkuRvgfNbucOr6pb2+rXACcBDgS+2B8C7gVOS7A9cA+y9pgcqSZIkSdLGYtJkvqouAp42Ir7bOOULOHCcdccDx4+IrwCePCL+Q2D3ydooSZIkSdJsMq175iVJkiRJ0swzmZckSZIkqWdM5iVJkiRJ6hmTeUmSJEmSesZkXpIkSZKknjGZlyRJkiSpZ0zmJUmSJEnqGZN5SZIkSZJ6xmRekiRJkqSeMZmXJEmSJKlnTOYlSZIkSeoZk3lJkiRJknrGZF6SJEmSpJ4xmZckSZIkqWdM5iVJkiRJ6hmTeUmSJEmSesZkXpIkSZKknjGZlyRJkiSpZ0zmJUmSJEnqGZN5SZIkSZJ6xmRekiRJkqSeMZmXJEmSJKlnJk3mkzwkyXlJvp3k0iTvbPEdkpyb5Mokn0yyWYs/uC2vbOvnDdR1SItfkWSPgfjiFluZ5OCB+Mh9SJIkSZI0m02lZ/4uYLeqeiqwAFicZBHwHuCoqpoP3Ars38rvD9xaVY8HjmrlSLIjsA+wE7AY+FCSTZNsCnwQ2BPYEdi3lWWCfUiSJEmSNGtNmsxX5462+KD2KGA34NMtvgx4UXu9pC3T1u+eJC1+clXdVVXfA1YCu7bHyqq6qqp+BpwMLGnbjLcPSZIkSZJmrSndM9960C8EbgKWA/8D/Kiq7m5FVgHbttfbAtcCtPW3AY8ajA9tM178URPsQ5IkSZKkWWtKyXxV3VNVC4Dt6HrSnzSqWHvOOOvWVvx+khyQZEWSFatXrx5VRJIkSZKkjca0ZrOvqh8B5wCLgC2SzGmrtgOub69XAdsDtPWPAG4ZjA9tM1785gn2MdyuY6tqYVUtnDt37nQOSZIkSZKk3pnKbPZzk2zRXj8UeC5wOXA2sFcrthQ4tb0+rS3T1p9VVdXi+7TZ7ncA5gPnAecD89vM9ZvRTZJ3WttmvH1IkiRJkjRrzZm8CNsAy9qs85sAp1TV55NcBpyc5F3At4DjWvnjgI8lWUnXI78PQFVdmuQU4DLgbuDAqroHIMnrgDOATYHjq+rSVtdbx9mHJEmSJEmz1qTJfFVdBDxtRPwquvvnh+N3AnuPU9cRwBEj4qcDp091H5IkSZIkzWbTumdekiRJkiTNPJN5SZIkSZJ6xmRekiRJkqSeMZmXJEmSJKlnTOYlSZIkSeoZk3lJkiRJknrGZF6SJEmSpJ4xmZckSZIkqWdM5iVJkiRJ6hmTeUmSJEmSesZkXpIkSZKknjGZlyRJkiSpZ0zmJUmSJEnqGZN5SZIkSZJ6xmRekiRJkqSeMZmXJEmSJKlnTOYlSZIkSeoZk3lJkiRJknrGZF6SJEmSpJ4xmZckSZIkqWdM5iVJkiRJ6hmTeUmSJEmSesZkXpIkSZKknpk0mU+yfZKzk1ye5NIkr2/xw5Jcl+TC9njBwDaHJFmZ5IokewzEF7fYyiQHD8R3SHJukiuTfDLJZi3+4La8sq2ftzYPXpIkSZKkPppKz/zdwJur6knAIuDAJDu2dUdV1YL2OB2grdsH2AlYDHwoyaZJNgU+COwJ7AjsO1DPe1pd84Fbgf1bfH/g1qp6PHBUKydJkiRJ0qw2aTJfVTdU1QXt9e3A5cC2E2yyBDi5qu6qqu8BK4Fd22NlVV1VVT8DTgaWJAmwG/Dptv0y4EUDdS1rrz8N7N7KS5IkSZI0a03rnvk2zP1pwLkt9LokFyU5PsmWLbYtcO3AZqtabLz4o4AfVdXdQ/H71NXW39bKS5IkSZI0a005mU/ycOAzwBuq6sfAMcDjgAXADcD7x4qO2LzWID5RXcNtOyDJiiQrVq9ePeFxSJIkSZLUd1NK5pM8iC6RP6mqPgtQVTdW1T1VdS/wr3TD6KHrWd9+YPPtgOsniN8MbJFkzlD8PnW19Y8AbhluX1UdW1ULq2rh3Llzp3JIkiRJkiT11lRmsw9wHHB5VX1gIL7NQLEXA5e016cB+7SZ6HcA5gPnAecD89vM9ZvRTZJ3WlUVcDawV9t+KXDqQF1L2+u9gLNaeUmSJEmSZq05kxfhmcDLgYuTXNhib6ObjX4B3bD3q4FXA1TVpUlOAS6jmwn/wKq6ByDJ64AzgE2B46vq0lbfW4GTk7wL+Bbdlwe0548lWUnXI7/PAzhWSZIkSZI2CpMm81X1NUbfu376BNscARwxIn76qO2q6ip+OUx/MH4nsPdkbZQkSZIkaTaZ1mz2kiRJkiRp5pnMS5IkSZLUMybzkiRJkiT1jMm8JEmSJEk9YzIvSZIkSVLPmMxLkiRJktQzJvOSJEmSJPWMybwkSZIkST1jMi9JkiRJUs+YzEuSJEmS1DMm85IkSZIk9YzJvCRJkiRJPTNnphsgSZIkSZqaeQd/YcL1V7/7heupJZpp9sxLkiRJktQzJvOSJEmSJPWMybwkSZIkST1jMi9JkiRJUs+YzEuSJEmS1DMm85IkSZIk9YzJvCRJkiRJPWMyL0mSJElSz5jMS5IkSZLUMybzkiRJkiT1zKTJfJLtk5yd5PIklyZ5fYs/MsnyJFe25y1bPEmOTrIyyUVJdh6oa2krf2WSpQPxXZJc3LY5Okkm2ockSZIkSbPZVHrm7wbYA3FlAAAQU0lEQVTeXFVPAhYBBybZETgYOLOq5gNntmWAPYH57XEAcAx0iTlwKPAMYFfg0IHk/JhWdmy7xS0+3j4kSZIkSZq1Jk3mq+qGqrqgvb4duBzYFlgCLGvFlgEvaq+XACdW5xvAFkm2AfYAllfVLVV1K7AcWNzWbV5VX6+qAk4cqmvUPiRJkiRJmrWmdc98knnA04Bzga2r6gboEn7g0a3YtsC1A5utarGJ4qtGxJlgH5IkSZIkzVpTTuaTPBz4DPCGqvrxREVHxGoN4lOW5IAkK5KsWL169XQ2lSRJkiSpd6aUzCd5EF0if1JVfbaFb2xD5GnPN7X4KmD7gc23A66fJL7diPhE+7iPqjq2qhZW1cK5c+dO5ZAkSZIkSeqtqcxmH+A44PKq+sDAqtOAsRnplwKnDsT3a7PaLwJua0PkzwCen2TLNvHd84Ez2rrbkyxq+9pvqK5R+5AkSZIkadaaM4UyzwReDlyc5MIWexvwbuCUJPsD1wB7t3WnAy8AVgI/AV4BUFW3JPlb4PxW7vCquqW9fi1wAvBQ4IvtwQT7kCRJkiRp1po0ma+qrzH6vnaA3UeUL+DAceo6Hjh+RHwF8OQR8R+O2ockSZIkSbPZtGazlyRJkiRJM89kXpIkSZKknjGZlyRJkiSpZ0zmJUmSJEnqGZN5SZIkSZJ6xmRekiRJkqSeMZmXJEmSJKlnTOYlSZIkSeoZk3lJkiRJknrGZF6SJEmSpJ4xmZckSZIkqWdM5iVJkiRJ6hmTeUmSJEmSesZkXpIkSZKknjGZlyRJkiSpZ0zmJUmSJEnqGZN5SZIkSZJ6xmRekiRJkqSeMZmXJEmSJKlnTOYlSZIkSeoZk3lJkiRJknrGZF6SJEmSpJ4xmZckSZIkqWcmTeaTHJ/kpiSXDMQOS3Jdkgvb4wUD6w5JsjLJFUn2GIgvbrGVSQ4eiO+Q5NwkVyb5ZJLNWvzBbXllWz9vbR20JEmSJEl9NpWe+ROAxSPiR1XVgvY4HSDJjsA+wE5tmw8l2TTJpsAHgT2BHYF9W1mA97S65gO3Avu3+P7ArVX1eOCoVk6SJEmSpFlv0mS+qr4C3DLF+pYAJ1fVXVX1PWAlsGt7rKyqq6rqZ8DJwJIkAXYDPt22Xwa8aKCuZe31p4HdW3lJkiRJkma1B3LP/OuSXNSG4W/ZYtsC1w6UWdVi48UfBfyoqu4eit+nrrb+tlb+fpIckGRFkhWrV69+AIckSZIkSdKGb02T+WOAxwELgBuA97f4qJ7zWoP4RHXdP1h1bFUtrKqFc+fOnajdkiRJkiT13hol81V1Y1XdU1X3Av9KN4weup717QeKbgdcP0H8ZmCLJHOG4vepq61/BFMf7i9JkiRJ0kZrjZL5JNsMLL4YGJvp/jRgnzYT/Q7AfOA84Hxgfpu5fjO6SfJOq6oCzgb2atsvBU4dqGtpe70XcFYrL0mSJEnSrDZnsgJJPgE8G9gqySrgUODZSRbQDXu/Gng1QFVdmuQU4DLgbuDAqrqn1fM64AxgU+D4qrq07eKtwMlJ3gV8CziuxY8DPpZkJV2P/D4P+GglSZIkSdoITJrMV9W+I8LHjYiNlT8COGJE/HTg9BHxq/jlMP3B+J3A3pO1T5IkSZKk2eaBzGYvSZIkSZJmgMm8JEmSJEk9YzIvSZIkSVLPmMxLkiRJktQzJvOSJEmSJPWMybwkSZIkST1jMi9JkiRJUs+YzEuSJEmS1DMm85IkSZIk9YzJvCRJkiRJPWMyL0mSJElSz5jMS5IkSZLUMybzkiRJkiT1jMm8JEmSJEk9YzIvSZIkSVLPmMxLkiRJktQzJvOSJEmSJPWMybwkSZIkST1jMi9JkiRJUs+YzEuSJEmS1DMm85IkSZIk9YzJvCRJkiRJPWMyL0mSJElSz0yazCc5PslNSS4ZiD0yyfIkV7bnLVs8SY5OsjLJRUl2HthmaSt/ZZKlA/Fdklzctjk6SSbahyRJkiRJs91UeuZPABYPxQ4Gzqyq+cCZbRlgT2B+exwAHANdYg4cCjwD2BU4dCA5P6aVHdtu8ST7kCRJkiRpVps0ma+qrwC3DIWXAMva62XAiwbiJ1bnG8AWSbYB9gCWV9UtVXUrsBxY3NZtXlVfr6oCThyqa9Q+JEmSJEma1db0nvmtq+oGgPb86BbfFrh2oNyqFpsovmpEfKJ93E+SA5KsSLJi9erVa3hIkiRJkiT1w9qeAC8jYrUG8WmpqmOramFVLZw7d+50N5ckSZIkqVfWNJm/sQ2Rpz3f1OKrgO0Hym0HXD9JfLsR8Yn2IUmSJEnSrLamyfxpwNiM9EuBUwfi+7VZ7RcBt7Uh8mcAz0+yZZv47vnAGW3d7UkWtVns9xuqa9Q+JEmSJEma1eZMViDJJ4BnA1slWUU3K/27gVOS7A9cA+zdip8OvABYCfwEeAVAVd2S5G+B81u5w6tqbFK919LNmP9Q4IvtwQT7kCRJkiRpVps0ma+qfcdZtfuIsgUcOE49xwPHj4ivAJ48Iv7DUfuQJEmSJGm2W9sT4EmSJEmSpHXMZF6SJEmSpJ4xmZckSZIkqWdM5iVJkiRJ6hmTeUmSJEmSesZkXpIkSZKknjGZlyRJkiSpZ0zmJUmSJEnqGZN5SZIkSZJ6xmRekiRJkqSeMZmXJEmSJKlnTOYlSZIkSeoZk3lJkiRJknrGZF6SJEmSpJ4xmZckSZIkqWdM5iVJkiRJ6hmTeUmSJEmSesZkXpIkSZKknjGZlyRJkiSpZ0zmJUmSJEnqGZN5SZIkSZJ6xmRekiRJkqSeeUDJfJKrk1yc5MIkK1rskUmWJ7myPW/Z4klydJKVSS5KsvNAPUtb+SuTLB2I79LqX9m2zQNpryRJkiRJG4O10TP/nKpaUFUL2/LBwJlVNR84sy0D7AnMb48DgGOgS/6BQ4FnALsCh459AdDKHDCw3eK10F5JkiRJknptXQyzXwIsa6+XAS8aiJ9YnW8AWyTZBtgDWF5Vt1TVrcByYHFbt3lVfb2qCjhxoC5JkiRJkmatB5rMF/ClJN9MckCLbV1VNwC050e3+LbAtQPbrmqxieKrRsQlSZIkSZrV5jzA7Z9ZVdcneTSwPMl3Jig76n73WoP4/Svuvkg4AODXf/3XJ26xJEmSJEk994B65qvq+vZ8E/A5unveb2xD5GnPN7Xiq4DtBzbfDrh+kvh2I+Kj2nFsVS2sqoVz5859IIckSZIkSdIGb42T+SS/kuRXx14DzwcuAU4DxmakXwqc2l6fBuzXZrVfBNzWhuGfATw/yZZt4rvnA2e0dbcnWdRmsd9voC5JkiRJkmatBzLMfmvgc+3X4uYAH6+q/0xyPnBKkv2Ba4C9W/nTgRcAK4GfAK8AqKpbkvwtcH4rd3hV3dJevxY4AXgo8MX2kCRJkiRpVlvjZL6qrgKeOiL+Q2D3EfECDhynruOB40fEVwBPXtM2SpIkSZK0MVoXP00nSZIkSZLWIZN5SZIkSZJ6xmRekiRJkqSeMZmXJEmSJKlnTOYlSZIkSeoZk3lJkiRJknrGZF6SJEmSpJ4xmZckSZIkqWdM5iVJkiRJ6hmTeUmSJEmSesZkXpIkSZKknjGZlyRJkiSpZ0zmJUmSJEnqGZN5SZIkSZJ6xmRekiRJkqSeMZmXJEmSJKlnTOYlSZIkSeoZk3lJkiRJknrGZF6SJEmSpJ4xmZckSZIkqWdM5iVJkiRJ6hmTeUmSJEmSesZkXpIkSZKkntngk/kki5NckWRlkoNnuj2SJEmSJM20OTPdgIkk2RT4IPA8YBVwfpLTquqymW2ZJEmSJG145h38hUnLXP3uF66Hlmhd26CTeWBXYGVVXQWQ5GRgCWAyL0mSJElrYCoJ/9rglwbr1oaezG8LXDuwvAp4xgy1RZK0nkz2IWNtfThYGx9m/KAirRn//u7PczJ9ayspnW3nTRuHVNVMt2FcSfYG9qiqV7XllwO7VtVfDpU7ADigLf4GcMV6bejatxVw80w3QjPKa0BeA/IakNeAvAbkNTA7Pbaq5k5WaEPvmV8FbD+wvB1w/XChqjoWOHZ9NWpdS7KiqhbOdDs0c7wG5DUgrwF5DchrQF4DmsiGPpv9+cD8JDsk2QzYBzhthtskSZIkSdKM2qB75qvq7iSvA84ANgWOr6pLZ7hZkiRJkiTNqA06mQeoqtOB02e6HevZRnPLgNaY14C8BuQ1IK8BeQ3Ia0Dj2qAnwJMkSZIkSfe3od8zL0mSJEmShpjMb0CSvDHJpUkuSfKJJA+Z6TZp3UtyfJKbklwyEHtkkuVJrmzPW85kG7VujXMNvC/Jd5JclORzSbaYyTZq3Rp1DQysOyhJJdlqJtqm9WO8ayDJXya5on0+eO9MtU/r3jj/FyxI8o0kFyZZkWTXmWyj1q0k2yc5O8nl7W/+9S3u50KNZDK/gUiyLfBXwMKqejLdhH/7zGyrtJ6cACweih0MnFlV84Ez27I2Xidw/2tgOfDkqvpN4LvAIeu7UVqvTuD+1wBJtgeeB1yzvhuk9e4Ehq6BJM8BlgC/WVU7AUfOQLu0/pzA/f8deC/wzqpaALyjLWvjdTfw5qp6ErAIODDJjvi5UOMwmd+wzAEemmQO8DDg+hluj9aDqvoKcMtQeAmwrL1eBrxovTZK69Woa6CqvlRVd7fFbwDbrfeGab0Z598BgKOAtwBOcLORG+caeC3w7qq6q5W5ab03TOvNONdAAZu314/Az4Ybtaq6oaouaK9vBy4HtsXPhRqHyfwGoqquo/vG/RrgBuC2qvrSzLZKM2jrqroBun/YgUfPcHs0s14JfHGmG6H1K8kfAtdV1bdnui2aMU8AfifJuUm+nOTpM90grXdvAN6X5Fq6z4mO0polkswDngaci58LNQ6T+Q1Eu/dlCbAD8GvAryR52cy2StJMS/J2umF3J810W7T+JHkY8Ha6YbWaveYAW9INt/1r4JQkmdkmaT17LfDGqtoeeCNw3Ay3R+tBkocDnwHeUFU/nun2aMNlMr/heC7wvapaXVU/Bz4L/PYMt0kz58Yk2wC0Z4dWzkJJlgK/D7y0/B3R2eZxdF/ufjvJ1XS3WVyQ5DEz2iqtb6uAz1bnPOBewIkQZ5eldJ8JAT4FOAHeRi7Jg+gS+ZOqauy993OhRjKZ33BcAyxK8rD2rfvudPfJaHY6je4/cNrzqTPYFs2AJIuBtwJ/WFU/men2aP2qqour6tFVNa+q5tEldTtX1Q9muGlav/4d2A0gyROAzYCbZ7RFWt+uB36vvd4NuHIG26J1rOUAxwGXV9UHBlb5uVAjxc6eDUeSdwJ/Qjek9lvAq8YmvdHGK8kngGfT9bbcCBxK9wHuFODX6b7o2buqRk2OpY3AONfAIcCDgR+2Yt+oqtfMSAO1zo26BqrquIH1V9P92omJ3EZqnH8HPgYcDywAfgYcVFVnzVQbtW6Ncw1cAfwj3S0XdwJ/UVXfnKk2at1K8izgq8DFdCNxAN5Gd9+8nwt1PybzkiRJkiT1jMPsJUmSJEnqGZN5SZIkSZJ6xmRekiRJkqSeMZmXJEmSJKlnTOYlSZIkSeoZk3lJkiRJknrGZF6SJEmSpJ4xmZckSZIkqWf+P2Wr6rd1auOGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1224x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(Gnew.dw, bins=100)\n",
    "plt.title('weighted degree distribution: DeepSphere V2\\nneighbors=50', fontsize=14)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/gusset/miniconda3/envs/PDMsphere/lib/python3.6/site-packages/ipykernel_launcher.py:24: RuntimeWarning: Mean of empty slice\n",
      "/home/gusset/miniconda3/envs/PDMsphere/lib/python3.6/site-packages/numpy/lib/nanfunctions.py:1434: RuntimeWarning: Degrees of freedom <= 0 for slice.\n",
      "  keepdims=keepdims)\n",
      "/home/gusset/miniconda3/envs/PDMsphere/lib/python3.6/site-packages/ipykernel_launcher.py:26: RuntimeWarning: Mean of empty slice\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "([<matplotlib.axis.XTick at 0x7f8116e2b320>,\n",
       "  <matplotlib.axis.XTick at 0x7f8116e2bf98>,\n",
       "  <matplotlib.axis.XTick at 0x7f8116ef20f0>],\n",
       " <a list of 3 Text xticklabel objects>)"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA+wAAAFSCAYAAABohlWkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAAIABJREFUeJzs3Xl829d55/vPIcGdFDdIpMRFCylLsmVblijJlpTFaT1xk2ayOI6ldtqbdEnbmXunW256p7dLptukM52ZTNubdiZtk06ahraz2amzNIvtRpK1ULIt27Jkkdq4iJLAneIGAuf+cX4gfqBIiqRAASS/79eLL4k44A8HIADi+Z3zPI+x1iIiIiIiIiIi6SUj1RMQERERERERkZspYBcRERERERFJQwrYRURERERERNKQAnYRERERERGRNKSAXURERERERCQNKWAXERERERERSUMK2EVE5sgYk2+M+Yoxps8YY40x61I9J1l+jDEvGGP+8hbX+agxZvBOzWmaOfyTMeYLqZzDZMaYTxljrnqv34+mej4yd97v8PVUz0NEZKEpYBeRJckY8wXvw7g1xoSNMeeNMX9mjClIwuF/Dng7sA9YDbQm4ZgyA2PMRWPMJ5J8zDsSzC7g7XwI+A++20n6Y5TujDEfN8Y8b4zpne3JM2PMVuD3gV/GvX6fXNhZygL5M+AdqZ4E6OSBiCysQKonICKygL4P/AyQBbwN+BugAPiV+RzMGJNtrR0D6oE3rbWvzXdixpgMwFhrI/M9hixv1truVM8hxhgTACLWWrtAx8+y1oanGMoH/hl4BvjvszxcvffvN25nvjPMSRaQ771zEEjp7hERkTtBK+wispSNWms7rbWt1tp/BL4EfCA2aIy52xjznDFmwBhzzRjzZWNMpW/8C9523t8yxrQBbcaYF4BfBd7urei94F231Bjz98aYHmPMsDHm+8aYe3zH+qgxZtAY8x5vJWYM2DLpNjq9bfafNsZkeKs217zLf8t/x4wxv2GMOWWMuWGMaTfG/I0xpmSK2/sxY8zr3vWeN8asn3Sc9xpjjnpz7jLGfNMYk+uNZRtj/tQY0+b9/HFjzLtnesCNMTnGmM94241HjDFHjDH7fOPv9B63H/Nud8gY02SM2T7DMV8A1gL/JbZrwje2xxjzonecdmPMXxljVvjG3+7NYdB7bI8aY7YaY94JfB4o8O3E+NQ0t19sjPmi97sY8XZr/Nqk8f/ljQ9482mI3d853E6nMeYJ3/eHvOMFvO83ej9fFXtcjLclfqbHyBuf8XkwxVxqjTFf925/wBjzNWNMtW/8U97xPmqMaQFGvfuY7z2nB73nwG9PcewZn1e+58h7jDHHjDFjwJTPO2vtZ6y1/wk4ONP98c8b+Lr3bTT2OHmvt981xrQaY0aNMa8ZY97v+7l13pwOGGN+aIwZBn5pmtvINsb8iTHmknes88aYf+8bf7v3PBzxHqP/bozJnjR+03PWG4u9rn/CGHPGe94/6z0HP2yMOef9zBeNMXm+Y874upzmfhhjzCeNMS3GvT+8Zoz5N77xn/F+f5t9l33aewxLve8ves+Vf/Dm3Wkm7QKZ6fUz6T5Pfu9MWNU2838vne3tT/kaMi6l4veBe0z8Nf7RmR5bEZE5sdbqS1/60teS+wK+APzTpMv+HAh5/18NhIA/BbYA9wHfBI4BGb5jDOAC/a3AvUAZ8HfAYaASKPOu+wxwBrdV/l7gWdxW+Txv/KPAuPdze4G7gCLvNvqBvwY2AweAKPAd4D951/tlwAI7fPfl14B3Aetw20JPAV/0jX8UCON2Gezy7t/LwHd913nUm9MfAXd71/kEkO+Nfwk44t2nDcD/ifuwfP8Mj/v/AK4A7/Ue18/hVsFWe+Pv9O7LMeBh7z5/F3gTt2o21THLvMfyP3qPeaV3+b3esX8T2AjsBl4CvuKNB4Ae3NbZOu+2fsqbVzbuxMuN2DGBwmlu/y+AV7zHcZ13Hx73xgwuWHzOG68H/tD7na6e4+08Cfy19/98XBAcAh70LvtF4Jzv+i8Af3mLx+iWz4Mp5mGAk7jn6k6gwXseNMV+R8CnvPv0z8B23OsjAHwWaMcF2FuBp73H4gu+48/4vPI9R14D/pV3nZW3eL03eD+z7hbXKwR+wbuu/3H6dW+eP4V7zf0BEAG2eePrvJ+5CHwYWA9UT3MbXwbagMe8uT8M/Kw3VuU9bn+Nex7+JNAJ/NdbPWen+H3uAB4COoDv4d6/7vNurwf4zdm+Lqe5H38MnMW9T6z35nEDeK/vOv/oPZ+yvd9bGHjYN37Re1z/X+9x/SXvd/2h2bx+bvHe+Sng9Unv+XN6L53D7U/7GgLyvN/XGeKv8bxU/e3Tl770tfS+Uj4BfelLX/paiC8mBezeB60Q8KT3/R8AP5j0M6Xeh7ldvmNcB3ImXe8vgRd832/0fu7tvsuKgT7gF7zvP8qkoNt3G61Apu+yJuDUpOtdBD4xw/19FBfgZUy6vU2+6/y092E5dp1DQOM0x6vzPuzWTrr8G8Bnp/mZAu/4P+u7LBNoAf7I+/6d3rze7bvOXu+yKQOg6e4/8L+Bv5102TbvWKtwQawF3jHNMT8KDM7iufQs8Plpxt6FC3zyJl3+CvDJOd7OrwBnvf8/gjuJ8ffAf/Au+xLwOd/1X8AL2Gd4jG75PJhiHo/ggtV1vss2eM+HH/e+/xQuiKnwXafQew7+9KTLevEC9tk8r3zPkcdu9Zj5fn5WAbt33Q8DdtJl7cDvTbrsBeAfvP+v847/m7c4duy94NFpxv8YaPY/9t7vaBR3kmY2z9nJv88/835fQd9lX8B7/2MWr8spbqcAGAbeNunyzwDf8n1f7D3v/hfufexPJ13/IvC9SZf9DXBwjq+fqd47P8XNAfuc3kvnePszvZcmzEVf+tKXvpL5pRx2EVnKHjWu2FcAl8f+DPB/eWM7cNvap8qBrMOtAIP7EDZ6i9vZggtCXopdYK3tM8a8hlu5jhnHfRCc7LRNzGW/igtymHTZqtg3xph34QqObcF9aM7ErXJV4lbcwKUEnPUdowP3OJQA3cADuA+5U9mOW306bYzxX54D/HCan6nzjn8odoG1NmKMeYnExwHcjgD/vPDuX9s0x57KDqDe+LaRe3MGqLPWvmRcdfLvGmN+APwAeNpaO9cigX8FfMW4bfvfA75prX3RN4d84PqkxykX93jMxQvAZ40xa3BB6/PAcWA/boXwHcBvTffDM7jV82CyLUCHtfZi7AJr7XljTAfu9/h97+I2a+1V38/V4Z6D/tfBoPc6iJnL86rp1nft9hmXQrEG3/PWcxB4zxzn9ADuveD5aca3AC9Za6OTbicbqLfWnprFc3by7/Mq0GmtDU26LPaam8vrMuZu3HP4OyYxvSILF/DGjtPnbf9+Hvfe9jtTHOulKb7/kPf/2b5+pnvvnGyu76Wzvf25voZERJJGAbuILGX/AnwctxLYYRMLRGXgtkFOVVXbH4TcmMXtmBnG/B92R+3UReYmF66y01yWAWCMWYub++eA3wO6cIHQl3Ef/GPGp5nLbOqXZHjX3znFXIan+ZnY42CnGJt8WXiKsbnWVcnArdZNVWysHcBa+zFjzGdwOxD+NfDHxpgPWGu/O9sbsdZ+23vMfwL4MeA5Y8zT1tqPeXO4iitqOFn/XO6MtfZNY8xVXLD+Ttxq5nHgL4wxd+O2U78wl2N65vo8MEz9O2TS5ZNfGzO9DmLm8ryazWsvmWbzvL3VnG71GNzysZ3Fc3aq3+e07xfM7XUZE/vZ9wGXJ41Nvq234Vb4K4AVuPej2Zrt62e6987J5vReOofbv533UhGR26KAXUSWsiFrbfM0YyeBjwCX7O1Xej6N++D2EO4kQWzV7l5cwbFka8AF5r8e+xBrjPnJeRznZVwA+rlpxgwux3e61cLJmnHbRPcB5715ZeIel3+cx/z8xnC7CPxOAvfM8DsGwFr7KvAq8KfGmG8D/wcub36qY053jBDwReCL3jG+bIz5ZW8OFUDUWnt+DnOfzou4POMG4EVr7TVjTAj4JNBsrW2f4WfncjszOQ1UGWPWxVbZjTEbcKvQp2f4uWZccPQg8d9/AS6XvcW7znyeVwvKWtvv7R7YR+Iq/z5mvr9TOYl7L3gYlzs92WngI8aYDN8q+z7c7y72GM30nJ2P+bwuT+O26a+11k63owZjzC7gd3Er5r+Dey/50KSrPTjF9296/5/N62chJev2k/XaExG5ic4Mishy9f/htpI/aYzZbYzZYIz5ca9acNFcDmStPYfbbv8/jTFvM8bcC/wDboXmdgPVqZzDvX//mjFmvTHmAK4I3Vz9MfC4MeaPjKuYf48x5teNMfnW2rdwOdNfMK769AZjTIMx5hPGmMkfyAGw1t7AbR//tFfReYv3fQWuGNntuAi8zRhTZYwJepf9KbDLGPPXxpgHjDH1xpifNMb8TwDvsfm0cZXk1xpjHsYVjDrtO2auMeYRY0zQGJM/1Q0bY/7AGPMB46q0b8EFJOe9VInv47YaP2Nc5e71xpiHjDH/0RjztrncjucF4Alccblr3mUvAv+GW6+uT/UYzcf3ccHil4wxO7yK2V/CBTfTBm/Wtdn6W1yQ+YhxXRL+Dl8gM5/n1UyMMZXGmG24gmIAdxtjthljyuZ4qP8CfMK4KvB3GWP+ALfq+l/nchDvveAp4G+MMY95z4e3GWN+xrvKZ3EnPj5rjNlijHkv8GlcLYKhWTxn52w+r0tr7QAuN/7PjDE/5722thljftkY83EAY0wh7nf519baZ3F53f/KGPPzkw73oDHmP3ivn18Efpb4rpjZvH4WUrJu/yKw1hiz3XuN5yzIbEVkWVLALiLLkrW2A1fsLFZF+A1cED/qfc3Vx3B57896/+bjCk9Nt3183qy1p3CVx38D90H+F5h6a/+tjvMt4IO4rd4v4wLDh3GPCbj79HngP+MqIP8TrrL3pRkO+1u4gOXzuJzT+3CPw5W5zm+S3wNqcKuQ1735n/Lms86b+6u4XO9YSsMQLpB7GngLV8DtS7hAH2vtYVxF6S97x/zkNLc9iju58Sruw30Rbqsw1lqLy3P+IW518ax3/zfh5ebP4XbA5QJnkhicT3XZVG56jObDu08f8I7xgnf7ncAHvLGZfMK7/te9f1/H23XiM5/n1XR+Gffc/ZL3/XPe9/96jsf5c1zQ/p+9OX8QV/RuNnnTk/0s7kTdn+Pu3xdwJwfxdkj8BC7X/RXcCY0vA7H2dzM+Z2/DfF6Xv4srpvYJ3Pvj93CV7y944/8Dt7L8Se++ncO9L/0PY8xG33H+G/HK6n+EK+73Fe9nbvn6WUhJvP2vAt/C1Ry4jqtQLyKSFObWf3tFRERERObGGHMRt3vgz1I9FxGRxUor7CIiIiIiIiJpSAG7iIiIiIiISBrSlngRERERERGRNKQVdhEREREREZE0pIBdREREREREJA0FUj2B+QgGg3bdunWpnoaIiIiIiIjInJ04cSJkrV15q+styoB93bp1NDU1pXoaIiIiIiIiInNmjLk0m+tpS7yIiIiIiIhIGlLALiIiIiIiIpKGFLCLiIiIiIiIpCEF7CIiIiIiIiJpSAG7iIiIiIiISBpSwC4iIiIiIiKShhSwi4iIiIiIiKQhBewiIiIiIiIiaUgBu4iIiIiIiEgaCqR6AkvZ4I8OEuntJau6iuzqajKDQYwxqZ6WiIiIiIiILAIK2Ofp4s/8LMPHjyfteLnbtrHuy/+ogF5EREREREQAMNbaVM9hzhoaGmxTU1Oqp3FL0aEhwh0djLW1EW5vJ9zWTtj7/1h7O9G+voTrZ+Tnk1VdTVZVlffvGrJ932cWFaXonoiIiIiIiEiyGGNOWGsbbnU9rbAvoIz8fHLq68mpr59yPDIw4AL5dhfIj7XF/z909CjRoaHE4xUXuyC+auqgPiM//07cLREREREREbkDFLCnUGZREZmbN5O7efNNY9ZaIr29blW+vZ1wu7cy39bG6PnzDP7oR9iRkcTjlZVNCuJjgX0VWWvWkJGTc6fumoiIiIiIiNwmBexpyhhDoLSUQGkpefduvWncWkskFPKC+PjKfLi9jZHTpxn4/g8gHE74mcCqVb4t91VkT6zSV5FVWYnJyrpTd09ERERERERuQQH7ImWMIbByJYGVK8nbtu2mcRuJMH79ejxnvq1tYrV++MQJ+p97DqLR+A9kZJBVWenbap8Y1AdWrcJkZt7BeygiIiIiIrK8KWBfokxmpgvAKyuh4eZaBjYcJnz1qhfEtyUUxrtx+DDj166BvyBhVhZZq1eTXV3lgvmq6oTt92pZJyIiIiIiklwK2BdS0+ehvwNqd0NVA+SVpHpGE0xWFtnV1WRXVwO7bxqPjo15ufMdE6v04fY2xtrbGfnh80S6uhKPl5MTz5evqopXt6+qJqu6isySEgX0IiIiIiIic6CAfb4+/x64dCh5x1u7Fz72reQd7zZlZGeTs349OevXTzk+U8u64VdPqWWdiIiIiIjIbVLAPl+zDa5HB6H9BLQe9b6Ow6gXzOYHoWa3W4Gv2Q3hEcjKXbg5J9EdaVkXy6FXyzoREREREVmGjPXnKS8SDQ0NtqmpKdXTmJ9oFEJnXfB+2Qviu1vcWEYWrNnmgvfYV1FFaue7AGZqWRdu7yDc3q6WdSIiIiIismQZY05Ya28uNjb5egrY08CNELQeg9Yj7t/2kxAZdWMlaxNX4VfdDRlLu1r7TC3rxtrbCXdcUcs6EREREZHlLjwMV16FtuPQ1gQ/+d8hvyzVs5oVBeyL2fgYdJ6Cy0fiW+kHr7qx7CKo3gE1D0LNLqhugNzi1M73DpupZV24rY1wZ+fsWtZ5/1fLOhERERGRNLVEa4cpYF9KrIXey748+KNw9Q2wUcC4VffYCnzNLihdD8u4IvtMLevC7e1qWSciIiIiku5G+tzO47Ymt4Le3gRDXqeqrAKo2u4WL6t3uo5ciyyVWAH7Ujc64IrZxfLg247DaL8bK1jlAvdYHvzq+xdNMbs7YdqWdV5Ar5Z1IiIiIiJ3UDQC1896W9u97e3XzwBerBq8ywXmsQB95RbIXNz10xWwLzexJ3ksD771KHSfd2OZ2bDmgcQgvnBVauebxmZqWTfW3j77lnXeZWpZJyIiIiLiM3jdrZjHgvP2kzA24MZyS7zg3AvQq7ZDXmlq57sAFLCLeyG0HfNy4Y9Bx8vxYnal6+J58DW7YdWWJV/MLllmalkXbmtTyzoRERERkZjxMbj6Wnxre9tx6LnoxkwmVNzjC9B3QnndskjvVcAuNxsfdVUUY3nwl4/CjWtuLGeFO4MVy4OvaoDcFamd7yJ0Oy3rEnLo1bJORERERBYba6GvLb5y3nbcxR+xRcPCSqjxBeert0H28ly8UsAut2Yt9F6K58G3HoNrvmJ2Fff4esLvcqvyy+Bs10JSyzoRERERWTLGbkDHK4m554Odbiwzx6XlVjfEc89XVCme8Chgl/kZ6Xf5JLE8+Nbj8XySwoqbi9kFtPqbTGpZJyIiIiJpKRqF7hZfcH4crp4GG3HjpesTc88rtkIgO7VzTmMK2CU5ohG49mZ8Bb71KPRccGMTxex8q/AqZregbq9l3c2F8dSyTkRERESmNNzjulK1+YrDjfS6sewiqN7h0mhjAXpBMLXzXWQUsMvCGbjqitnFgviOlyEy5sZK10Otr5jdys0qZncHqWWdiIiIiMxZZByunU7MPe865w0aV6A6tq29eqdrs6bP+LdFAXsasNYuj2BnfNTlrsSK2bUehRvX3VjOCveijq3AVzdAjtqcpYpa1omIiIgIA52JK+cdJyHsdTrKD/p6njfAmu0qRr0AFLCngd/++muc7uhnX32QPfXl7FhbSk5gGZyJstZtm2/1rcJffQOwYDJuLmZXslbFJ9LEbbWsm1wYTy3rRERERFIvPAKdpxILw/W1urGMAFTel5h7rkLTd4QC9gX2xP98iaMXupN2vN3ry3jylx5K2vHSzkife3OIBfFtTb5idpWTitndp2J2aUgt60RERETSnLWux3n7iXiAfuUURL0uRMU18a3tVQ3uc3dWXkqnvFwpYE8j/SNhjp7v5lBziEPNIc5dGwSgJD+LPXXl7KkLsq8+yNry/OWxhR68YnanJxWzu+jGMnOgans8iK/eBYUrUzpdubWktqyrriarokIt60RERERmMjoA7ScTc8+HQm4skOc+U/sD9BWrUztfmaCAPY1d7R/hcEuIg+e6ONwS4kqfW5WsKsljb305e+uD7KkLsrJoma0+DnQmbqO/8kq8mF1ZXXwL/UQxu4zUzlfmRC3rRERERG5DNAqhtxK3tl87DXjxXPlGX+75Tlh1N2QGUjplmZ4C9kXCWsv50A0ON4c41OwC+P6RcQA2VRSxtz7I3vpydm8opzBnmb3gwiMuaI8F8JePxM8Y5hRDja+YXdUOFbNb5NSyTkRERMTnRhe0N8UD9PaTMNrvxnKLfS3VdrqV9Pyy1M5X5iQtA3ZjzK8CvwgY4HPW2s8YYz7lXeaVFee3rbXfmuk4SylgnywStbzR0cfB5hCHm7s4drGbsfEogQzD/TUlLoCvK+eB2lKyA8tshdla6D6fuAofO6toMqBi66RidrUqmLGEqGWdiIiILFmRMHS+5lbNY0F693k3FivaPBGcN0B5vXabLnJpF7AbY7YCjcAuYAz4DvArwE8Dg9baP5vtsZZywD7ZSDjCyUs9HGwOcaili9faeolayMvKZNf6sokK9FsqV5CRsQyDj+Fe96aWUMzO1QigaHViMbvK+yCQndr5yoJRyzoRERFZNPraE7e2X3kFxr3ivYUViVvbV2+DnMLUzleSLh0D9seBd1trf8H7/neBUSAfBeyz1jcc5sj5Lg41hzjYHOL89RsAlBVk81BdOfvqg+ytC1JbvkzbaUXGJxWzOwK9l91YINf1kZwI4ndBQTC185U7Ri3rREREJCXGhlxA7u97PtDhxjKzXUDu73teXKNdostAOgbsW4BngIeAYeAHQBPQBXwU6Pe+/01rbc9Mx1rOAftkV/qGXe67F8BfGxgFoKYsj711Qa+AXTnlhcusgJ1f/xVoOxZfhe94Jd7aorw+sZhdcJO2Fy1DalknIiIiSRFL4ZxYPT8Ona+DjbjxkrW+nuc7oXKr2hkvU2kXsAMYY34e+HfAIHAaF7h/Ggjhyhv+IbDaWvtzU/zsx4GPA9TW1u64dOnSnZr2omGtpeX6IAfPue3zR1q6GBh1Bey2rF7Bvvpy9tQH2bWujILlVsDOLzzsgnb/KvyQl/+cW+zayCUUs9MWpOVOLetERERkSsO9rue5v+/5sLf2mF3otVXz5Z6rVbF40jJgT7hhY/4EaLPWftZ32Trgn6y1W2f6Wa2wz854JMpr7X1e//cuTlzqYSwSJSvT8EBNKXvq3Rb6+2tKyMpcxqvKE8Xsjrqvy0fh+ptuzGS6M58Txex2Q3G1tilJgjm3rMvMJKuiIp4/P6kwnlrWiYiIpKFoBK69mdjzPHTWGzSwclM877x6p9eGWH/PZWppGbAbY1ZZa68ZY2qBf8Ztj8+11l7xxn8d2G2t3T/TcRSwz8/wWISmS90TFehf7+jDWijIzmT3hnL21JWzb2OQTRVFqp493ANtJ7wg/oj7f9jVC6BoTWIxu9X3QaZWS2V68ZZ1/q32alknIiKS1gavJRaGaz8Z/zyYV+bb2t7gVtJzi1M7X1lU0jVg/xFQDoSB37DW/sAY80VgG25L/EXgl2IB/HQUsCdHz40xjpzvcgF8SxcXQu4NKFiY44J3rwJ9damKa7lidm/E+8G3HoO+WDG7PPcm7Q/i1QdT5kAt60RERFJsfNRrq+bLPY8VLs4IuPbB/gC9bIN2XMptScuAPVkUsC+M9t5hb/u820IfGnQF7NaV57OnPsi++iAPbSintECt0QDo7/By4L08+CuvQtTVDKB846RidnepmJ3M2+22rJsojKeWdSIiIm5XW+9l38p5k/scFxlz4yuqEre2r74fsvJSO2dZchSwy22x1vLW1cGJAP7ohW4GR8cxBu5Zs2KiAv3OdWXkZSs3B/CK2b0cz4NvPQrD3W4st8QL3r0AvmoHZBekdr6yZKhlnYiIyAxGB91nNH/u+Y1rbiyQB2segOod8cJwxVWpna8sCwrYJanCkSin2no51Oy20L98uYdwxJKdmcH2tSUugN8Y5L6qYgLLuYCdn7XQ1RLPg289BtfPuDGTCZX3xlfhax90xexEkizpLeuqqsjI1i4bERFJU9EodJ3zbW0/4dIarVf8tazO1/N8J1Tco1pEkhIK2GVBDY2Nc+xCN4dbujh4LsTpK/0AFOUE2L2hnL1eBfr6VYXKpfUb7nFndluPulz49hMQ9lY/V1Ql5sFX3qs/ILLg1LJOREQWtaHuxJZqbSdg1EsVyyme1FZtBxSUp3a+Ih4F7HJHdQ2O8tL5Lg41d3GoOcTlbheErirKYW99kD115eytD7KmRPk/CSLjcPX1eB586zHoa3VjgTz3hyW2Al+9U8Xs5I5TyzoREUkbkTBcfSMx97yr2Y2ZDFh1d2LueflG1RCStKWAXVKqtXvI5b+3dHG4OUTXDVfEY0OwgL31QfbWl/PQhiDF+VqJu0lfO7Qdi+fBd56KF7ML3uVbhX8Qyuv1h0hSSi3rRERkwfRfSWyr1vEyjA+7sYKVUL0rnnu+5gHIUVFVWTwUsEvaiEYtZ68OcKg5xMHmEMcudDM0FiHDwNaqYvZ6Feh3rC0lN0srbzcZG4oXs4t9Dfe4sbxS98cqtgq/Zjtkq2CYpA+1rBMRkVkJD7tK7f6t7f1tbiwjy1Vq9+eel9SqrZosagrYJW2NjUd5pbV3ogL9K629jEct2YEMdq4rZU+dC+C3VhWTmaE34ptY67Z/xfLgW49B6Kwbywj4itl5X6p0KmlMLetERJYha6HnQrxie9tx1wM9tqOwpDZesb16p/tsk5Wb2jmLJJkCdlk0BkfHOXYhnv9+pnMAgBW5AR7yct/31AWpW1mglbXpDHV7xey8AD6hmF11fBt97W6o2KpidrJoqGWdiMgSMNIH7SfjAXp7Ewx5O6yyCrzCcA3xIL2oIrXzFbkDFLCuODmbAAAgAElEQVTLonV9YJTDLSEOey3k2ntdrlLlilz2eNXn99YHqVihM63TioRdMbtYHnzrsfi2sqz8eDG7mgfdH0gVs5NFaMaWdd5ldnQ04Wcyy8u9rfZqWScisiCiEbh+NjH3/PoZwIs5gpu8re1e7vnKLZAZSOmURVJBAbssCdZaLncPcbDZBfCHWkL0DrkWU/WrCtnnVaB/sK6cFblaNZ5RX1s8eG89CldOgY24seCmeB58zW5XzE67GWSRU8s6EZE7YPC6WzGPBejtJ2Fs0I3llsQrtlc3uAWDvJLUzlckTShglyUpGrWcvtI/UYH+2IUuRsJRMgzcV13iAvj6cnasLSUnoAJ2Mxq74YrZxfLgW4/CSK8byyvzVuC9Vfg1D6iYnSw5alknIjJH42Nw9bXE3POei27MZELlVl/P8wYor9MCgMg0FLDLsjA6HuHly/ECdq+29RGJWnKzMti5rsy1kKsLcveaFSpgdyvRqFfM7kh8JT70lhvLCEDlffE8+JrdsGJNaucrssDm1bJuzWovX14t60RkkbPW7c6LbWtvO+6quEe8VKPCSqjZGQ/QV2/TyX2ROVDALsvSwEiYo+e73Rb6lhBvXXVbskrys3hoQ7nXAz7IuvJ8fXCejaHu+Op7rJhdrP9pcU18Bb5ml1fMTjlosnyoZZ2ILCmxnXcTq+dNMNjpxgK5LiCPFYarboAVVVo9F7kNCthFgGv9Ixxu6fJy4EN09I0AUFWSx566cvZtDPJQXTmrilTAblYiYdd2JdYP/vJRGOhwY7FidrE8+OoG1ydeZJlSyzoRSVvRKHS3+ArDHYerp+O1bUrXJ+aeV2yFgIpyiiSTAnaRSay1XAjd4FBLF4fOhXjpfBd9w67g1KaKookK9Ls3lFOYo5XiWetrS8yD73wt/gd/5WbfKvxu5bKJ+MynZV2sPZ1a1onInAz3uF1y/tXzWN2a7KJ4xfaqBhegFwRTO1+RZUABu8gtRKKWNzr6Jvq/H7/Yzeh4lMwMw7aaEvZ6PeAfqC0lO5CR6ukuHmM33IcCf0X6EW8lMa8sMQ9+zQOQlZfa+YqkIbWsE5F5i4zDtdOJuedd57xBA6u2+La274TgXZChgpkid5oCdpE5GglHOHmph0MtIQ42d/FaWy9RC3lZmexaX8beehfAb6lcQYYK2M1eNOo+KPhX4WMfHDICsPr+eB58zW5YsTq18xVZBJLSss63/V4t60QWsYHOxJXzjpMQ9nbo5Afj29qrd7oT5bkrUjtfEQEUsIvctr7hMEfOd3G4OcTB5hAt128AUFaQzUN1bvv83rogteXahjpnN7qgbXIxO1dfgOJaX0/4XbDqHhWzE5mj221ZF1i1CpOVhQlkYgIBCAQwmQFMVsB9n+n+NYFMNzbpi4TrZmICk47lv27s/xnaySRyS+ER6Dzlyz1vgr5WN5aRBZX3Juael65TKppImlLALpJknX0jE+3jDrWEuNrvtqPWlOWxt85Vn99TV055YU6KZ7oIjY8lFrNrPQoDV9xYVoHLravxF7MrSe18RRapiz/zswwfP57qaUwtI8P1sc/KcgF85lQBfiYEJo1neScTAgEIxE4OBLxjeT8XO8GQNdXJhix3rKypTkzc4mSD77omMza3zJtPRmRqu7HMg7Wux3lbE7TH2qqdgqi3e6a4JnFre+V9kKUiuiKLhQJ2kQVkraXl+iCHml0F+iPnuxgYGQdgc2WRW32vD7JrfRkFKmA3d9a6FYOJlnKxYnZRwLhidrE8+JrdULZBKwgiSWajUYhEsOPjE1/E/h+JYMPh+Hh4HCLjvutGsONh7/ruOhPj4XFsxHcs77ru+PHbIxK77s3Hct/75hYZh4nr3jyO7z7YSATGx+/sg2lMYrAfO5mQ6Qv+pzzZkDXliYkZTzZMe2Jiil0SSTgxQWam2hEmy+gAtJ9MzD0fCrmxQB5UbY8H6FUNSiETWeQUsIvcQeORKK+197kWcudCnLjUw1gkSiDDsL22dKIC/f01JWRlatvnvIwOesXsvCC+7Vi8mF1+eTx4r9kNa7apmJ2ITMtam3AyIiGgH4+AdwLBnZgYd997/5/qZMNNJxMmrnvrkw0TJybG/def7mSDd2JiPH5y4qaTEZNqF9wRN52MmH6XxC1PTPjHZ3ViYoqTDVOlb2TdYhdE7OSDb+4EAgt3MiIahdBZX3De5ArF4X0uL9+YmHu+6m6lh4ksMQrYRVJoeCxC06XuiQr0r3f0YS0UZGeye0P5RA/4TRVFWpmYr9iHHX81+q5mN5aR5YrZ1fqK2RVVpna+IiJ3wMTJiMknG27aCeE7mRA7wTA+3a6JW+yCGPedyAhPfeLh5pMN4YRdEVOeuEjYNTGempMRmVMH91OfbJicvuEbN1EY68OMdGNGumA4hLGjGAMmKxuKV2NKqzGlNZiyWkz+iqnTN6Y9MeGduMjynWyYqlaE/6SKPn+IpJQCdpE00js0xkstXRxqCXGouYsLIVfALliYzZ664EQF+upSFbC7LTdCvm30x1yl3Fgxu5LaxGr0FfeojY2IyCJz086CW56Y8O1UmObExJQnGyaue/OJCXfyYJoUjIg7sWCH+rHD/diRQRgdwobHsFGDtQZMAEsAS6bL9Bp3J1gSCkHeCZmZC78LYlYnG+ZRxPJWJyZUxFIWAQXsImmsvXeYQ80hrwJ9F6FBV8BubXk+e73q8w/VlVNWoL7Jt2V8zFXTjeXBXz4Kg51uLLvQbTWs2e2C+OqdkFuc2vmKiMji09eeWLX9yivxk8WFFYlb21dvg5zCKQ9jo9EpUjSmSMGY4cRE/Pqzqx1xO+kcc6kdccdPRqiIpSwCCthFFglrLeeuDXLwXIjDLSGOnO9mcHQcY+Du1SvYVx9kT32QXevKyMvWm/RtsRZ6LycWs7v6eryY3aotvlz4XSpmJyIiicaGXEA+EaCfgIEON5aZ7QJyf4BeXK2/I3hFLP27ImZzYsK/K2JOJxtuPnExc+2IydedW+0IIpE7+2BOV8Ryul0QKmKZthSwiyxS4UiUU219Ey3kTl7uIRyxZGdm8EBtyUQAf391MQEVsLt9owOJxexaj8OoV8yuYGU8eK/Z7T6IqWWOiMjyYC10n/cF58eh83WwXoBWus5Va59oq7YVAmrtutxYa2/uoOEL7hM6aviLWk51ssG/K+JWJyaSUTtiiqKVk09M3HG3WcSy8nd+h+yamjs/73lQwC6yRAyNjXP8Ys9EAP9GRz8ARTkBdm8oc1vo64NsXFW4rM5KLphoFK6f8RWzO+I+sEF89SQWwNfshqKK1M5XRESSY7jXncD19z0f7nFj2YVeW7Wd8bZqhStTO1+RBTZtR41b1Y7w74qYXDtilicb/LsgBn74QyKhUNLuV97Onaz74v9O2vHmSwG7yBLVfcMVsDvY7LbQX+oaAmBlUQ5768onAvg1JWprljSD110buVgefMfLEHF1ByhZm1iNftXdKmYnIpLuohG49mZiz/PQWW/QwMrN3rZ2bwV95Wa9t4tIUilgF1kmWruHONziitcdbg7RdWMMgA3BAi94L+ehDUGK87NSPNMlZHwUrviK2bUehcGrbiy7aFIxuwYVsxMRSbXBa4mF4dpPQth1bCGvLL5yXt3gVtL1vi0iC0wBu8gyFI1azl4dmNg+f/RCN0NjEYyBe6uKJyrQN6wrJTdLKwVJYy30XornwV8+Ctfe8BWzuxtqfcXsSterCJGkrefOP8c3z3+TA5sOsK9qH5laVZTFZnwUOl9LzD3vvezGMgJQea8v97xBBUZFJCUUsIsIY+NRXm3rnQjgX77cy3jUkh3IoGFt6cT2+XurisnM0IeVpBrp9xWzO+JWdEZd/QEKViXmwa/ZpkJFckd87Dsfo+lq8v5+NlQ08PlHP5+044nMWaz7h39re+cpiLjdZqyoim9rr94Jq++HLKWMiUjqKWAXkZsMjo5z/EI3B70A/kznAAArcgM8uKGcfRuD7KkLUreyQAXski0aSSxmd/kI9FxwY5nZsOaBxCC+cFVq5yvLWjga5geXf0DjmUZOXD1BTmYOj657lAObD3BP8J5UT0+Ws9FBV0fEH6DfuObGAnnuvTSWe17VAMVVqZ2viMg0FLCLyC2FBkc53NLFoXMhDjaHaO8dBqByRS576svZ563AV6xQK7MFMXgtsSd8x8vxVaHS9Ykt5VZtUcEjSYm3et7iyTNP8s3z32R4fJh7g/eyf/N+3r3u3eRkameILKBoFLrOJeaeXzvtpRsBZXWJPc8r7oFM1WsRkcVBAbuIzIm1lsvdQxxq7uKQV4G+ZygMQP2qwokK9A/WlbMiVx+IFsT4KFx51cuDP+KC+djKUc6KxGJ2VQ2QuyK185VlZWBsgGdbnqXxTCMX+y9SmlPKBzd+kI9s+ghVhVrFlCQY6vbaqsUC9BMw2ufGcoqheke8pVp1A+SXpXa+IiK3QQG7iNyWaNRy+kr/RAX64xe6GQ5HyDBwX3UJe+tdAL+9VgXsFoy10HMxcRX+6huABZMBq+7xbaPfBaXrVDhJFpy1liNXjtB4ppEX2l4A4O3Vb+fApgM8uOZBMkxGaicoi0Mk7N7P/Fvbu1vcmMlwxTr9ueflGyFDzy0RWToUsItIUo2OR3j5ci+Hm932+Vfb+ohELTmBDHatL2NPXZB99UHuXrNCBewW0kg/tDe5SvStR90H3TFXi4DCisQ8+NX3q5idLKgrg1d4+q2n+eq5r9I90s3aFWt5YtMTvL/+/azI1g4Q8enviAfmbU0uBWjcpWFRsBKqd8Vzz9c8ADlFqZ2viMgCU8AuIgtqYCTM0fPdHGpxBezeujoIQHFeFnvqytlT7wL4deX5KmC3kKIRuPZmYk/4notuLDNnimJ2K1M6XVmaxiJjfPfid2k828ip66fIC+TxnvXv4cDmA2wq25Tq6cmdFh526T3+3PP+djeWmQ2V9yXmnpfUaneQiCw7CthF5I661j/iCth5Feg7+kYAWFOcO9E+bk99OauKVMBuwQ1chbZj8Tz4K6/Ei9mVbUgsZrdyi7aZSlKd7jpN45lGvnXhW4xGRtm+ajv7N+/nx2t/nCwVBFt6rHUdLyZWz4+7HujRcTdeUhvf1l690/VA184fEZH0DNiNMb8K/CJggM9Zaz9jjCkDngTWAReBj1hre2Y6jgJ2kfRmreVi1xAHm0Mcbg5xuKWLvmFXwO6uikL2eqvvu9aXUaQCdgsvPBIvZhf7unHdjeUUJxazq27QVlRJir7RPr7R/A0azzTSNthGMC/IYxsf4/G7HqeioCLV05P5GumD9pPxAL29CYa63FhWAVRtj6+cVzVAkX7XIiJTSbuA3RizFWgEdgFjwHeAX8EF8N3W2k8bY/4foNRa+1szHUsBu8jiEolaTnf0uwC+JcSxC92MjkfJzDBsqymZqED/QG0p2QGt9i642IpYq28V/tppJorZVdwT30Jfs1vbVeW2RG2Ug+0HaTzTyMH2g2SYDN5V+y72b9rPzsqdSplJZ9EIXD/jKwzX5L7H++wY3JS4tX3lZsgMpHTKIiKLRToG7I8D77bW/oL3/e8Co8DPA++01l4xxqwGXrDWzpjwpoBdZHEbCUc4ebnH2z7fxam2XqIW8rIy2bW+bKIC/ZbKFWSogN2dMdLnPoy3+ovZuboEFFZOUcwuO7XzlUWpdaCVp84+xdfOfY3+sX7qiuvYv3k/76t7HwVZBamengxedyvmsa3t7Sfj7wO5Jb6t7Q1QtQPySlI7XxGRRSwdA/YtwDPAQ8Aw8AOgCfgZa22J73o91trSmY6lgF1kaekbDnP0vJf/3tJF8zX3AbGsIJuH6srZ61Wgry3PT/FMl5FoxK26x1bgW49C7yU3lpnjtr36g/iCYGrnK4vKyPgI377wbb585su82f0mBVkFvG/D+ziw+QAbSjakenrLw/iYyzX3B+ixgpUmEyq3xgP0qgYor9NOGxGRJEpqwG6M2QC8A5dnngdcB04Ch6y1I3OY1M8D/w4YBE7jAvePzSZgN8Z8HPg4QG1t7Y5Lly7N9mZFZJHp7Bvx+r+HONzcRWe/e5upLs1jX32QPfVB9tSVEyxU4aI7aqAzsSd8xysQdbUJKKuL58HXPui2yqqYndyCtZZToVM0nmnkuxe/SzgaZlflLvZv3s/DNQ8TyND26qSwFvraEnueX3kVIqNuvGh1Ys/z1dsgWydIRUQWUlICdmPMTwO/CjQAV4EOXJBdBtQBI8CXgD+11s4pgjbG/AnQ5h1fW+JFZErWWlqu33AB/LkQL53vYmDEVR/eXFnEPq8C/a71ZRTk6MP9HRUecRXo/avwQyE3llMMNTvjQXxVA+QUpna+kta6hrv4evPXeersU1y5cYWK/Aoev+txHrvrMYJ52sExJ2M3XJ9zf9/zwU43Fsh1Abk/QC+uSu18RUSWodsO2I0xLwPjwBeAZ621rZPGc3Db2/cDjwH/1lr79C0mtcpae80YUwv8s/fzvw10+YrOlVlrPznTcRSwiyxf45Eor3f0T7SPa7rUw9h4lECG4YHakokWcttqSsjK1ArvHWUtdJ/3VaM/5nrETxSz2xrfQl+7G4prtMVWbhKJRnix7UUazzTy0pWXCGQEeGTtIxzYfIBtK7epSN1k0Sh0t/h6nh+Hq6fBRtx46frE3POKrapBISKSBpIRsL/XWvvcLG8sCKy31h6/xfV+BJQDYeA3rLU/MMaUA08BtcBl4HFrbfdMx1HALiIxI+EITRd7JirQv9beh7VQkB0rYOcC+E0VRSpglwrDvTcXswvfcGNFq3158A96/ZkVSEjchb4LPHn2SZ5pfobB8CCbyzbzxKYneM/695CftUy3bA91e23VvJZqbU0w0uvGsougeocv93yH6kuIiKSptCs6l0wK2EVkOr1DYxw53zWR/34+5ILDYGE2D9UF2Vdfzp66IDVly/TDfqpFxl0xO39P+N7LbiyQC2smF7MrT+18JS0MhYf4p/P/ROPZRs71nKMou4gP1H+AJzY9wdoVa1M9vYUTe734c8+7znmDBlZtSdzaHrwLMjJTOmUREZmdBQvYjTE/AbwLyAQOWmu/Nr8pzp8CdhGZrY7e4Ynt84daurg+4IosrS3PZ49Xff6hunLKCrSymzL9V+Jb6FuPumJYsWJ25fXxPPiaB72ARKkOy5W1lpPXTtJ4ppHvX/o+43acvWv2sn/zft5W9TYyF3uwOtDp29p+AjpOQnjIjeUHE3uer3kAclekdr4iIjJvCxKwG2N+H5ez/k0gG/gw8LS19tfnO9H5UMAuIvNhreXctcGJAP7I+W4GR8cxBu5evWJi+/zOdaXkZ6uAXcqEh13BLH8QP9TlxnKLoXpXPA9+zXYVs1umrg9d5ytvfYWn33qa68PXqSqs4vG7HudDGz9Eae6M3WHTQ3gEOk/5AvQm6PPKBWVkwer7XLHGWJBeuk41H0RElpBkVYnfYq190/f9WWC7tfaG9/0DwA+stWVJmPOsKWAXkWQYj0R5ta2Pw82uhdzJyz2EI5asTMP22tKJFnL3VxcTUAG71LEWuloSi9ld9/40xfpFx7bQ1+yG4moFNstIOBrmh5d/SOOZRpquNpGdkc2j6x/lwOYDbA1uTfX0HGtdj/O2pnjf8yun4jtJimsSt7ZX3gdZuSmdsoiILKxkBexvAf8I/LG1NmyMeRH4PNCIW2H/TeCD1tr7kjPt2VHALiILYWhsnOMXeyYC+NNX+rEWinIC7N4QL2C3cVWhKlWn2nDPpGJ2J3zF7NbE8+Brd7vgJzMrtfOVO+JczzmePPskz7Y8y/D4MFvLt7J/834eXf8oOZk5d24iowPxwnCx3PNYy8OsfLczJFYcrqoBVqy+c3MTEZG0kKyAvQD4I+AR4JeB68A/ADsAC7wBfNRaezIZk54tBewicid03xjjpZYuDrW4LfSXulwu6cqiHPbWlU8E8GtK8lI8UyEyDldfj2+hbz0GfbFidnlQtT2eB1+zC/Lv6MYwucMGxwZ5tuVZGs82cqHvAiU5JXxw4wd5YtMTVBUmued4NAqhs4m559dO4z4mAeUbE3PPV90NmUq5ERFZ7pKaw26M2QX8NXAM+CTur1CGtbbvdic6HwrYRSQVWruHONwS4lBzF4dbQoQGxwDYECxgT305++qDPLihnJJ8FbBLC/0dUxSzG3dj5RvjK/A1u933Kma35FhrOdp5lMYzjTzf+jzWWt5R/Q72b97PQ2seIsPM43d+oyu+rb3tuFtJH+13Y7nF8VXz6p3uRJFODomIyBSSXnTOGJOJC9Z/Afi/U1EdPkYBu4ikmrWWs1cHOHguxOGWLo6c72JoLIIxcG9V8UQF+oZ1peRmLfLK1UvF2NDNxeyGu91Ybom3Au+twldth+yC1M5XkqrzRidPnX2Kr577Kt0j3dQW1fLEpid4f/37Kc4pnvqHxsfczo3Ytvb2Jug+78ZMBlTcE887r94JZXU68SMiIrOSrC3xAeDjwBbgVVz++nrcavsg8G+ttR1JmfEcKGAXkXQTjkR5tbV3ov/7ycs9jEct2YEMGtaWTmyfv7eqmMwM5b+nBWuhq3lSMbszbsxkQuW9iavwxdWpna8kxVhkjO9d+h6NZxp55for5Gbm8t4N72X/5v1szixKrNp+5RUYH3E/WFiRuLV99TZ1KBARkXlLVsD+98BO4FngbcAJa+2/98Y+Bvwe8F+stZ9NyqxnSQG7iKS7G6PjHLvYzaFzrv/7m1fcltmi3AAPbYjnv9etLFABu3Qy1J1YzK79RLwP9oqqxDz4yntVzG4xGxvizbeepfHc03yr/xwjWLaNjHKgf4BHRsbJWr0tMUBX9wEREUmiZAXsvcBD1to3jTF5wOvW2jrf+CrgM9ban0rGpGdLAbuILDahwVEOt3RNVKBv6xkGoGJFjgve61wAX1msVk5pJRJOLGZ3+Sj0t7mxQB5U7XDBe+2DLqhTvnJ6stZtZZ9YPT8Ona+DjQDQV7aWb6ys4UnbS2u4n/Lcch676zEev+txKgsqUzx5ERFZipIVsJ8FPgv8FfCvgN+z1u5K2iznSQG7iCx2l7uGONgc4lBLiMPNIXqGXD/mupUFE/3fH9xQTnGeVnDTTl+bF8B7QXznqXgxu+BdvlX43RDcqFXZVBjudbsj/Lnnwz1uLLvQ1SiI5Z1XNUDhSgCiNsrhjsM0nmnkX9r+hQyTwcM1D7N/8352Ve7SbhgREUmaZAXsj+DauAWBK8BHrLWHkzbLeVLALiJLSTRqebOzn8PNXRxsDnHsQjfD4QgZBu6tLmFffTl764JsX6sCdmlpbAg6Tk4qZucFh3mlUL0rnge/Zjtk56d2vktNNALX3kzseR466w0aWLk5vq29usF9n3Hr11HbQBtPnX2KrzV/jb7RPjYUb2D/5v28b8P7KMxW7rqIiNyepFWJN+50ctBaez1Zk7tdCthFZCkbG4/y8uUeDrV0cag5xCutvUSilpxABjvXlXn57+Xcs0YF7NJSNHpzMbtYAJkR8IrZeXnwNbuhOMl9wZe6gau+tmpNrq1a+IYbyyvzVW1vcCvpudNUgJ+lkfERvnPxOzSeaeSNrjfID+Tzvrr3cWDzAepK6m59ABERkSkkva1bOlHALiLLycBImGMXuicq0J+9OgBAcV6WK2C30bWQW1eery276Wqo2wWYsQC+rQnGXR0DVlTHV+BrdkHFVhWzixkfhc7XEnPPey+7sdjJj4mt7TugbMOCpiC8dv01Gs828u0L3yYcDbOzcif7N+3n4dqHycrQ70xERGbvtgN2Y8z3gU9Zaw/e4oZKgH8H9Ftr/2I+k50rBewispxdGxjhpZYuDp4Lcag5REefazu1pjh3ovr8nvpyVhWpgF3aioRdIBrbQt96FPrb3VhWvlfMzgviqxuWRzE7a10w7t/a3nkKImNufEWVb2v7Tlh9P2TlpWSq3SPdfO3c13j67NN03OhgVd4qPrzpwzx+1+ME84IpmZOIiCwuyQjYPwr8ITCMa+vWhMtjHwFKgbuBfcCjwDeAT1pr25Ix+VtRwC4i4lhrudg1xKFmF7y/dL6LXq+A3V0VhRMV6HdvKKMoVyuAaa2vLb4Cf/mIC+i9KuYEN/lW4XdDef3iL2Y3Ouhy/9ua4gH6jWtuLJAHax5IzD1fsSa1851CJBrhX9r+hcazjRzuOEwgI8AjtY+wf/N+Hlj1gHa8iIjItJJVdC4LeBw4gAvOY4lgFjgNfBf4nLX27NRHWBgK2EVEphaJWk539HOoxQXwxy50MzoeJTPDcH918UQF+gdqS8gJqIBdWhu74fKz/bnwI71uLK8sngNfs9sFt+lczC4aha5zvq3tTXDtNNioGy+rS+x5XnHPoksLuNh3kSfPPskzzc8wEB7grtK72L95P+9d/17ys9L4dyMiIimxIDnsxphiIA/ostaGb2N+t0UBu4jI7IyEI5y83DNRgf5UWy9RC3lZmexcX8a++nL21AW5e/UKMlTALr3Fgt5YAH/5qPsevHzu+1w/+Fggn8oV6aFur61aLEA/AaN9biynGKp3JOaeL6Et/0PhIZ678ByNZxp5q+ctirKKeH/9+3li0xOsK16X6umJiEiaUNE5ERG5Sd9wmKPnuzjc4gL45muDAJTmZ7GnLjhRgb62TAXsFoUbXb5idkddkDzuahpQXOPrCR8rZhdI/hwiYbj6RmLueXeLGzMZsOruxNzz8o2QkZH8eaQZay0vX3uZxjONfO/S9xi34+xZs4f9m/bz9uq3kzmL1nIiIrJ0KWAXEZFbuto/4uW/uxZynf0u2KsuzWNvXZC9G4PsqSsnWJiT4pnKrIyPwdXX4nnwrUdh4IobyypwK9v+YnZ5pXO/jf6OeGDe1gQdL8cr3hesdH3nYwH6mgcgRz3LQ8MhvvLWV3j6rae5NnSNNQVreHzT43xo44coy106uwtERGT2FLCLiMicWGs5HxyAo8wAACAASURBVLoxUcDucEsXAyPjAGyuLGJvvWsft2t9GQU5C7BSK8lnbWIxu9Yj0Pl6vJjdyi2JufDldYnF7MLDcOXVxNzzWDX7zGy3Dd+fe15Su/iL4S2gcDTM85efp/FsI8c7j5Odkc27172bA5sPcO/Ke1M9PRERuYMUsIuIyG2JRC2vt/e5/u8tIY5f7GFsPEogw/BAbQl76oLs2xhkW00JWZlLf4vzkhGrzh7Lg287BiNefnl+uQvciyrdynnnaxB1J20oqY1va6/e6XqgB7TzYr6ae5ppPNvIN1u+ydD4EPeU38P+zft5dN2j5AbUklFEZKlTwC4iIkk1Eo7QdLGHQy0hDjeHONXeh7WQn53J7vVlEz3gN1UUqYBduvr8e+DSoeQdb+1e+Ni3kne8ZWhwbJBvnv8mjWcaOd93nuKcYj5U/yE+sukjVBdVp3p6IiKyQJIasBtjPgP8jbX29WRM7nYpYBcRSb2+oTAvnXe574daQpy/fgOA8oJs9tQH2VtXzt76IDVlamklcivWWo51HqPxTCPPtz5P1EZ5W/XbOLD5AHvW7CHDaBeLiMhSkuyA/RDwIHAC+Bug0Vrbf9uznCcF7CIi6edK3/BE8bpDzSGuDYwCUFuWP5H//lBdOWUF2SmeqUh667zRydNvPc1X3/oqXSNd1BbV8pFNH+ED9R+gOKc41dMTEZEkSPqWeGPMJuDngH8DFANfA/7WWvvi7Ux0PhSwi4ikN2stzdcGOehVoD9yvovBUZcLfc+aFRPb53euKyU/WwXsRKYSjoT53qXv0Xi2kZevvUxuZi7v2fAe9m/az5byLamenoiI3IYFy2E3xmQA78EF7+8FLgN/C/wva233POY6ZwrYRUQWl/FIlFPtfRw657bPn7zUy1gkSlamYXtt6UQAf391MQEVsBO5yZnuMzSeaeS5888xEhnh/pX3c2DzAR5Z+wjZmdq1IiKy2CxkwJ4DPAb8PPAO4EWgElgLfNxa+49zn+7cKGAXEVnchsciHL/YPZH//kZHP9ZCYU6ABzeUTVSg37iqEKM2YSIT+kb7eKb5GZ48+ySXBy5TllvGYxsf4yObPkJlQWWqpyciIrO0EFviG3Cr6vuBIeDvcYXoLnjjvwr8trW2Yt6zniUF7CIiS0vPjTFeOt/lWsg1h7jYNQTAyqIc9taVuyJ29UGqSvJSPFOR9BC1UV7qeInGM4282PYixhgernmY/Zv3s7tyt050iYikuWQXnXsN2AR8F/gc8Jy1NjLpOiuBq9baBd/LqIBdRGRpa+sZ4nBz10QP+NDgGADrgwXsrS9nb50rYFeSr63AIu2D7Tx19im+du5r9I72sr54PU9seoL3172fwuzCVE9PRESmkOyA/XeBv7PWtidjcrdLAbuIyPJhreXs1YGJCvRHz3dxYyyCMbB1TbGX/17OznVl5GZlpnq6IikzGhnlOxe+Q+OZRl7vep38QD7vq3sf+zftp760PtXTExERn2QH7NlAhrV2ZNLluUDUWjs275nOgwJ2EZHlKxyJ8mpr70QA/3JrD+GIJTuQwY7aUvZtDLKnrpx7q1TATpav10Ov8+UzX+Y7F77DWHSMhooG9m/ez7tq30VWRlaqpycisuwlO2B/BnjRWvvfJl3+a8A7rbUfmPdM50EBu4iIxNwYHefYxW4ON4c42NzFm1f6ASjKDfDghnL2eSvwdStVwE6Wn56RHr7e/HWeOvsU7YPtrMpbxYfv+jAfvuvDrMxfmerpiYgsW8kO2EO4wPz1SZffAzxvrV0175nOgwJ2ERGZTtfgKIdbujjcEuJgc4jW7mEAKlbksLcuONFCrrI4N8UzFblzItEIP2r/EY1nGjnUcYiACfDja3+c/Zv3s33Vdp3MEhG5w5IdsA8B2621ZyZdvgU4aa29o2V7FbCLiMhsXe4a4lBLiEPNIQ63dNF9w2Vx1a0smAjeH9xQTnGetgnL8nCp/xJPnn2SbzR/g4GxATaWbmT/pv385IafJD8rP9XTExFZFpIdsB8Bvmut/f1Jl/8h8Ki1due8ZzoPCthFRGQ+olHLmc6Bif7vR893MxyOkGHg3uoS9nkV6LevLVUBO1nyhsJDfPvCt2k828iZ7jMUZhXy/vr388SmJ1hfvD7V0xMRWdKSHbC/F/gG8BTwQ+/iHwMe///Zu/OAqqv8/+PPyyIg7gruKCiLCwiaioKpkYpmlApy+VZj0zRuU5PTz7aZb7vNtPitJp0yv+rYd2q4iJVbVogZhlZGiUvKIop7brmAuAD3/P5A77iLil7E1+Mfu+fz+ZzP+/MR0/c957wPMMwYs+gaYr1iSthFRKQqnCyzk739kGP/99XbD1FuN3i4udC9bSNHBfpOLerj6qIpw1IzGWPI3pdNck4yS7YuocxeRq/mvbCGWOnbqi+uLvrySkSkqlVpwn6qw1jgv4GIU02rgVeMMZ9fdZRXSQm7iIhcD8Unyli15QCZ+RVr4HN+KQKgvpc7vQIaExXYhKh2jfFv4q01v1Ij7T+2n4/zPmZO3hz2luyluXdzRgaPZHjgcBp5NnJ2eCIiNUaVJ+xVwWKx/Al4GDDAOuC3wDSgL3D41GkPGmOyL9WPEnYREbkR9hWdYOWp9e8rNh1g56GKAnYt6nvSu30TottXbCHnW08F7KRmKbOX8fX2r7Hl2Pj+l+9xd3FnUNtBWEOshDUJ0xdWIiLXqNol7BaLpSWQCXQ0xhyzWCxzgMVAP2CRMWZuZftSwi4iIjeaMYatB0oqps8XVBSwO1RSCkBQ0zr0bleRwPcMaERdTxWwk5qj4FABthwbCzcv5GjpUTo27og12Mpg/8F4uunLKhGRq1HVa9hrAX8BkgA/4Kx/iRhjLru46VTC/h3QBThCxZr4d4D/Qgm7iIjcZOx2w4bdR8jcVDEC/0PhrxwvtePqYqFLq/qOCvQRfg3wcNMaYLn5HS09ysKChdhybBQcLqC+R32GtR/GyOCRtK7b2tnhiYjcVKo6YX8NSAT+BrxFxVr2toAVeNYY834lg3oMeAU4BqQZY+6zWCyzgV7ACWAp8LQx5sSl+lHCLiIi1c2JsnJ+2nrIUYF+zfZD2A14urvQw78xUe0aE9W+CR2b18NFBezkJmaMIWtPFsk5yXy17Svsxk50y2isIVaiW0bjYnFxdogiItVeVSfsW4BxxpgvLBZLERBujCmwWCzjgBhjTHwl+mgIfExF4n8ISAXmUpGk/wLUAqYDBcaYly5w/WhgNICfn1+3rVu3XjZuERERZzlyvJTvN/96av37fvL3FgPQsLY7vds1oXf7xkS3b4Jfo9paDyw3rV+O/sLcvLnMzZvLgeMHaFWnFYnBiQwLHEZ9j/rODk9EpNqq6oS9BAgxxmyzWCy7gaHGmB8tFos/sMYYU68SfSRQsWf77059/g0QaYwZf8Y5/YCJxpihl+pLI+wiInKz2XPkOCsL9jsq0O8+fByAlg28KorXtW9M73ZN8Knr4eRIRa5caXkp6dvSseXY+GnvT3i4ejDEfwjWECsdG3d0dngiItVOZRN2t0r2tw1ocerXTcAg4EcqprIfu4I+Ii0WS+1T18QAWRaLpbkxZrelYnjhXmB9JfsTERG5aTSt58mwiFYMi2iFMYbN+4+yctN+Mjft5/P1u0nJ2g5ASLO6jv3fe/g3po5HZf+qFnEed1d3BvsPZrD/YHJ/zcWWa+OzzZ/x6aZPCfMJwxpsZVDbQdRyreXsUEVEbiqVHWH/G1BsjHnFYrHEA8nADqAl8IYx5i+VupnF8iIVU+LLqNjH/WHgc8AHsADZwFhjTPGl+tEIu4iI1CTldsP6nYdZUXC6gN1BTpbZcXOxEN66gaOAXXjrBtRy0/pguTkcOXmE+Zvmk5KbwtYjW2nk2YjhgcMZGTSS5nWaOzs8ERGnuq7bulkslp5AFJBnjFl0FfFdEyXsIiJSkx0vLefHrQcd69/X7TyM3UDtWq708G9E9KkEPrhpXRWwk2rPbux8t+s7knOTWb5jOQD9WvXDGmIlsnmkajiIyC2pyhJ2i8XiDnwI/NkYU1BF8V0TJewiInIrOVxSyrebK9a+Z27az+Z9RwFo7F2L3u2bOCrQt25U28mRilzaruJdzMmdwyf5n3DwxEHa1muLNcRKXLs46taq6+zwRERumKouOncQ6GaM2VwVwV0rJewiInIr2334GCs2HXCsgd9bVLEbql+j2o71773bNaGRt9YLS/V0ovwEaYVpJOcks27/OrzcvBgaMBRriJWghkHODk9E5Lqr6oR9JrDRGDO5KoK7VkrYRUREKhhjKNhXTGb+flYUHOC7ggMUnSgDoGPzekQHNqF3u8b08G9E7VoqYCfVz8/7fyY5J5nPt3zOSftJujXthjXESoxfDO4u7s4OT0TkuqjqhP154E9ABpAFHD3zuDHmzauM86ooYRcREbmwsnI763YeZsWp0fefth7iZLkdd1cLEX4NT61/b0xYqwa4u6qAnVQfh44f4tNNn5KSm8LO4p34ePkQHxRPfFA8vrV9nR2eiEiVquqEfcslDhtjTMCVBHetlLCLiIhUzrGT5fxQ+KujAv3Pu45gDNTxcKOnfyNHBfqgpnVU/EuqhXJ7OSt2rSA5J5nMnZm4Wdy4w+8OkkKS6Na0m35ORaRGuK5V4p1NCbuIiMjVOXj0JN9uPuCoQF94oAQAn7oe9D5VvC6qfRNaNvBycqQisO3INlJyU/h006cUnSyifYP2JIUkMTRgKLXdVWRRRG5eSthFRETksnYcLGHlpgOnRuAPsL/4BC4WyH5+IPU8tX5YqodjZcf4fMvn2HJsbPx1I3Xc6xDXLo7EkEQC6t/QiZ4iIlWiqqfEv3Op48aYP15BbNdMCbuIiMiVS3z/W77f8muV9dfTvxEpY3pVWX8il2OMYc2+NdhybXxZ+CVl9jJ6Nu9JUkgSfVv1xc1FhRVF5OZQ1Qn7snOa3IEQwA34yRhzx1VFeZWUsIuIiIjc2vYf288n+Z8wJ3cOe0r20My7GSODRjI8cDiNvRo7OzwRkUu67lPiLRaLJzAT+MYYM+2qOrlKSthFREREBKDMXkbG9gySc5P5fvf3uLu4M7DtQKzBVrr4dFGROhGplm7IGnaLxdIR+NIY0/qqO7kKSthFRERE5FybD28mJSeF+QXzOVp6lA6NOpAUkkSsfyxebiqkKCLVx41K2PsC84wxDa+6k6ughF1ERERELuZo6VEWFSzClmtj06FN1KtVj2Hth5EYnEjrejd0nElE5IKqeg374+c2Ac2B+4CvjDH3XVWUV0kJu4iIiIhcjjGGrD1Z2HJsLN22FLuxE9UyiqSQJKJaROHq4ursEEXkFlXVCfuWc5rswD7gK+Bvxpiiq4ryKilhFxEREZErsbdkL3Pz5pKal8r+Y/tpWaclicGJDGs/jAaeDZwdnojcYrQPu4iIiIjIOUrLS1m6bSnJOcn8tPcnPFw9GOw/GGuIlU6NOzk7PBG5RVT1CHstwMUYc/ycdk/Abow5edWRXgUl7CIiIiJyrfIO5mHLsbFo8yKOlR0jrEkY1hArA9sOxMPVw9nhiUgNVtUJ+3wgwxjz5jntE4B+xph7rzrSq6CEXURERESqStHJIhYULMCWY6PwSCENPRoyPHA4I4NH0qJOC2eHJyI1UFUn7PupSMzXn9PeCVhmjPG96kivghJ2EREREalqdmPnu93fYcuxkbEjA4C+rfpiDbES2TwSF4uLkyMUkZqisgm7WyX7qw2UXaDdDtS9ksBERERERKojF4sLvVv0pneL3uwu3s2cvDl8kv8Jy7Yvo229tiQGJxLXPo56teo5O1QRuUVUdoT9O+BLY8zz57S/DMQaY7pfp/guSCPsIiIiInIjnCw/yZeFX2LLtbF231q83Ly4K+AurMFWghsFOzs8EblJVfWU+LuAecAcKrZyA4gBEoBhxphF1xDrFVPCLiIiIiI32s8HfsaWY+PzLZ9zovwEXX27khSSRIxfDO6u7s4OT0RuIlW+rZvFYokF/huIONW0GnjFGPP5VUd5lZSwi4iIiIizHDp+iHmb5pGSm8KO4h008WpCfFA88YHxNPVu6uzwROQmoH3YRURERESuI7uxk7kzE1uOjcydmbhYXLjD7w6SQpK4reltWCwWZ4coItVUVU+J7wtgjMm4QLsxxiy/2kCvhhJ2EREREalOth/Z7ihSd+TkEdo3aE9icCJ3t7sbb3dvZ4cnItVMVSfsPwEvGWPmndN+N/CCMabbVUd6FZSwi4iIiEh1dKzsGF9s+YLknGQ2/roRb3dv4trFYQ22EtAgwNnhiUg1UdUJ+1GgszFmyzntbYH1xpg6VxnnVVHCLiIiIiLVmTGGtfvXYsux8WXhl5TaS+nZrCfWECv9WvfDzaWyuyuLSE1U1Qn7fuAeY8yKc9qjgQXGmEZXHelVUMIuIiIiIjeLA8cO8OmmT5mTO4fdR3fTtHZTEoISGBE0giZeTZwdnog4QVUn7B8BfkCcMebgqbZGVGz1ttMYk3SN8V4RJewiIiIicrMps5exfMdyknOS+W73d7i5uDGwzUCSQpLo4tNFRepEbiFVnbA3B5YDvsDaU81hwF6grzFm1zXEesWUsIuIiIjIzWzL4S2k5KYwf9N8ikuLCWkUgjXYypCAIXi5eTk7PBG5zq7HPuy1gfuAcMAC/AT82xhTci2BXg0l7CIiIiJSE5SUlrBo8yJsuTbyD+ZTt1Zd7m1/L9ZgK371/JwdnohcJzdsH3aLxXKnMSb9mjq5QkrYRURERKQmMcbw096fSM5JZunWpZSZMqJaRpEUnER0y2hcXVydHaKIVKHrmrBbLJaWwG+B3wF+xpgb+n8QJewiIiIiUlPtLdnLx3kfk5qXyr5j+2hZpyUjg0cyvP1wGng2cHZ4IlIFrseUeFcgDvg9MICKtewpQOq5271db0rYRURERKSmK7WX8tW2r7Dl2Mjak0Utl1oM9h9MUkgSnZp0cnZ4InINqixht1gswcDDwG+Ao8C/gaeALsaYDVUQ6xVTwi4iIiIit5L8g/nYcmws3LyQY2XHCG0SijXEyqC2g/Bw9XB2eCJyhaokYbdYLN8AnYG5wIfGmIxT7aUoYRcRERERuaGKThaxoGABthwbhUcKaejRkGGBwxgZPJKWdVo6OzwRqaSqStjLgH8A/2uMWX9GuxJ2EREREREnMcbw/S/fY8uxsWz7MgBub3U7ScFJRLaIxMXi4uQIReRSKpuwu13m+G1UrFn/xmKxFAL/ByRfe3giIiIiInK1LBYLkc0jiWweye7i3aTmpfJx/sd8vf1r2tRrQ2JwIve0v4d6teo5O1QRuQaVKjpnsVg8gQQqqsJHAS7A08AMY8zB6xrhBWiEXURERETkbCfLT5K2NQ1bjo01+9bg5ebFEP8hJIUkEdwo2NnhicgZrtu2bhaLpT3/KULXGPjKGDO4ktf+6dS1BlhHxdZwzQEb0Aj4CXjAGHPyUv0oYRcRERERubiNBzZiy7WxePNijpcfJ8I3gqSQJO70uxN3V3dnhydyy7uu+7CfuoErMBR4yBhzTyXObwlkAh2NMccsFsscYDEwBPjEGGOzWCzTgDXGmPcu1ZcSdhERERGRyzt84jDzNs0jJTeF7UXbaezZmPigeBKCEmjq3dTZ4Yncsq57wn6lTiXs3wFdgCPAPGAK8BHQzBhTZrFYegEvGGMGXaovJewiIiIiIpVnN3ZW7FyBLdfGNzu+wcXiwh1+d2ANttK9WXcsFouzQxS5pVRV0bkqY4zZabFYJgPbgGNAGvAjcMgYU3bqtB2A9qMQEREREalCLhYX+rTqQ59WfdhetJ3U3FQ+2fQJS7YuoV39dlhDrNzd7m683b2dHaqInOFGjrA3BD4GEoFDQOqpz88bY9qfOqc1sNgYE3qB60cDowH8/Py6bd269YbELSIiIiJSEx0vO87nWz7Hlmtjw4ENeLt7c3fA3VhDrLRr0M7Z4YnUaNVxSnwCEGuM+d2pz78BelFRfV5T4kVEREREnMAYw7r967Dl2Pii8AtK7aX0aNYDa4iV/q374+ZywyblitwyqmPC3hOYBXSnYkr8bCALuB34+Iyic2uNMe9eqi8l7CIiIiIiVe/X47/ySf4nzMmdw+6ju/Gt7UtCUALxQfE08Wri7PBEaoxql7ADWCyWF6mYEl8GrKZii7eW/Gdbt9XA/caYE5fqRwm7iIiIiMj1U24vZ/mO5dhybazctRI3FzcGtBlAUkgS4T7hKlInco2qZcJeVZSwi4iIiIjcGIWHC0nJTWH+pvkUlRYR3DAYa4iVIf5DqO1e29nhidyUlLCLiIiIiEiVKSkt4bMtn2HLsZF3MI+67nW5p/09WEOstKnXxtnhidxUlLCLiIiIiEiVM8aweu9qbDk2lmxdQpkpI6pFFNYQK31a9sHVxdXZIYpUe0rYRURERETkutpXso+5+XOZmzuXvcf20rJOSxKCEhgeOJyGng2dHZ5ItaWEXUREREREbohSeynLti3Dlmvjh19+oJZLLWL9Y7EGWwn1CXV2eCLVjhJ2ERERERG54TYd3IQt18bCgoWUlJXQqXEnkkKSiPWPxcPVw9nhiVQLSthFRERERMRpik8Ws6BgAbZcG1sOb6GBRwOGBQ4jMTiRlnVaOjs8EadSwi4iIiIiIk5njGHVL6uw5dhYtn0ZdmPn9la3Yw2x0rtFb1wsLs4OUeSGq2zC7nYjghERERERkVuTxWKhZ/Oe9Gzek1+O/kJqXipz8+aSkZ6BX10/EoMTuaf9PdT3qO/sUEWqHY2wi4iIiIjIDVVaXsqSrUtIzkkme182nq6e3BVwF9YQKyGNQpwdnsh1pynxIiIiIiJS7eX8moMtx8Znmz/jePlxwn3CsYZYGdhmIO6u7s4OT+S6UMIuIiIiIiI3jcMnDjN/03xSclPYVrSNRp6NiA+KJyEogWbezZwdnkiVUsIuIiIiIiI3Hbux8+2ub0nOSWb5juW4WFzo37o/1hArPZr1wGKxODtEkWumonMiIiIiInLTcbG4ENUyiqiWUewo2sGcvDl8mv8p6dvSCagfQGJwInHt4qhTq46zQxW57jTCLiIiIiIi1dqJ8hN8seULbDk21h9YT2232tzd7m6swVbaN2zv7PBErtgtNyW+tLSUHTt2cPz4cSdFJXLjeHp60qpVK9zdVYhFREREbi3r9q3Dlmvjiy1fcNJ+ku7NumMNttLfrz/uLvq3kdwcbrmEfcuWLdStW5fGjRtrXYvUaMYYDhw4QFFREf7+/s4OR0RERMQpDh4/yCf5nzAndw67ju7C18uX+OCKInVNvJo4OzyRS7rlEvaNGzcSEhKiZF1uCcYYcnJy6NChg7NDEREREXGqcns53+z8BluOjRW7VuBmcWNAmwFYQ6xE+EYoP5Bq6ZYsOnfmH8a3luTx96X5Vdb3YzGB/GlAUJX1J3It9BePiIiISAVXF1f6te5Hv9b9KDxcSEpuCvM3zefzws8JahiENcTKXf53Udu9trNDFbliNWqE/UpHGxPf/xaAlDG9qiQuV1dXQkNDKS0txc3NjVGjRjFhwgRcXFyqpP/TFi1axLPPPovdbqe0tJTHHnuMMWPGXPT82bNnk5WVxdSpU6s0jjMVFhYSHR3Ntm3bznre8PBwpk+fzvHjx5kwYQJr167FZrMRHx9/3WK5VVzNz7yIiIjIraCktITFWxZjy7GRezCXuu51uaf9PSQGJ9K2fltnhydya46wO5uXlxfZ2dkA7N27l//6r//i8OHDvPjii1V2j9LSUkaPHs2qVato1aoVJ06coLCwsMr6v5Dy8nJcXV0veU7btm1p3bo133zzDX379gUgJyeHoqIievToQWFhIbNnz2by5MnXNVYRERERkdrutYkPimdE4Aiy92WTnJOMLdfGhxs/pFfzXiSFJHF7q9txdbn0v3FFnK1qh37FwdfXl+nTpzN16lSMMZSXl/PEE0/QvXt3wsLCeP/99x3nvvHGG472559/HqgYsQ4JCWHUqFGEhYURHx9PSUkJRUVFlJWV0bhxYwA8PDwIDg4G4MEHH2Ts2LH06dOHoKAgFi1a5LjHrl27iI2NJTAwkCeffNLRnpaWRq9evejatSsJCQkUFxcDFQn4Sy+9RHR0NKmpqRQUFBAbG0u3bt3o06cPOTk55z1zUlISNpvN8dlms5GUlOToLywsrMpnG4iIiIiIXIzFYiHCN4LXb3+dJfFLeCT8EQoOF/DHZX9kyCdDmLFuBr8e/9XZYYpcVI0cYX9x4c9s2HXksudt2F1xzump8ZfSsUU9nr+70xXFERAQgN1uZ+/evcyfP5/69evzww8/cOLECaKiohg4cCD5+fnk5+ezatUqjDHExcWxfPly/Pz8yM3NZebMmURFRfHQQw/x7rvvMnHiROLi4mjTpg0xMTEMHTqUpKQkRyJcWFhIRkYGBQUF9O/fn02bNgGQnZ3N6tWrHQn+o48+ipeXF5MmTSI9PR1vb29ee+013nzzTZ577jmgYuuwzMxMAGJiYpg2bRqBgYF8//33jB8/nq+++uqs5x05ciQRERFMmTIFNzc3UlJSSE1NvaJ3JiIiIiJyPTTxasKYLmP4XejvWLZ9GbYcG3//6e+8m/0usW1jsYZYCW0SqlpBUq3UyIS9OjldIyAtLY21a9cyd+5cAA4fPkx+fj5paWmkpaUREREBQHFxMfn5+fj5+dG6dWuioqIAuP/++3nnnXeYOHEiM2bMYN26daSnpzN58mSWLFnC7NmzgYqk2cXFhcDAQAICAhwj4TExMdSvXx+Ajh07snXrVg4dOsSGDRsc9zh58iS9ev1nPX9iYqIjppUrV5KQkOA4duLEifOetVmzZnTq1ImlS5fStGlT3N3d6dy5c5W9SxERERGRa+XmUlFFfkCbARQcKsCWY2NBwQIWbl5Ix8YdSQpJIrZtLJ5uns4OVaRmJuyVHQmv4Xs64gAAIABJREFU6qJz59q8eTOurq74+vpijGHKlCkMGjTorHO+/PJLnnnmmfOKxhUWFp737d6Zn0NDQwkNDeWBBx7A39/fkbBf7BoPDw9Hm6urK2VlZRhjGDBgAMnJyReM39vbGwC73U6DBg0c6/Mv5fS0+KZNmzqmw4uIiIiIVEftGrTjL5F/YUK3CSwsWIgtx8azK55lctZkhrcfzsjgkbSq28rZYcotTAuKr5N9+/YxduxYHnnkESwWC4MGDeK9996jtLQUgLy8PI4ePcqgQYOYNWuWY+34zp072bt3LwDbtm3j228rvlRITk4mOjqa4uJivv76a8d9srOzadOmjeNzamoqdrudgoICNm/e7FjffiGRkZGsWLHCMW2+pKSEvLy8886rV68e/v7+juntxhjWrFlzwT5HjBjB4sWLSUlJwWq1VvZ1iYiIiIg4jbe7N9YQK5/e8ymzBs2iR7Me/N+G/2PIJ0P4w9I/8M2Ob7Abu7PDlFtQjRxhd5Zjx44RHh7u2NbtgQce4PHHHwfg4YcfprCwkK5du2KMwcfHh3nz5jFw4EA2btzomIpep04dPvzwQ1xdXenQoQMffPABY8aMITAwkHHjxlFeXs7rr7/OmDFj8PLywtvb2zG6DhAcHEzfvn3Zs2cP06ZNw9Pz4lN5fHx8mD17NklJSY4p7pMmTSIo6Pz95j/66CPGjRvHpEmTKC0txWq10qVLl/POa9CgAZGRkezZswd/f39H+w8//MCwYcM4ePAgCxcu5Pnnn+fnn3++qvcsIiIiInI9WCwWujfrTvdm3fnl6C/MzZvL3Ly5jN8xntZ1W5MYnMi97e+lvkd9Z4cqtwjtw871mxJ/LQoLCxk6dCjr16+v9DUPPvggQ4cO1R7ntwjtwy4iIiJy/ZWWl5K+LR1bjo2f9v6Ep6snQwKGYA220qGx/i0mV+eW34f9rSV5/H1pfqXObfv0Z5c957GYQP404PyRZxERERERqbncXd0Z7D+Ywf6Dyf01l+ScZBZvWcwn+Z/QxacL1hArA9sMpJZrLWeHKjXQLT3CLnIz08+8iIiIiHMcOXmE+Zvmk5KbwtYjW2nk2YgRgSMYGTySZt7NnB2e3AQqO8KuonMiIiIiIiJXoF6tejzQ8QEW3LuA9+98nzCfMGaun8mgjwcxYdkEvtv9HTfjwKhUPzV2SryIiIiIiMj15GJxoXfL3vRu2ZudxTtJzU3l4/yPWbptKf71/UkMTuSedvdQp1YdZ4cqNymNsIuIiIiIiFyjlnVaMqHbBNIT0nkl+hXquNfh1VWvckfqHbz87cvkH6xcfS2RM9XcEfZlf4OMV6uuv75PQ/9nqq4/ERERERGpcTxcPYhrF0dcuzh+3v8zyTnJzNs0jzl5c+jWtBtJIUnc4XcH7i7uzg5VbgK3dtG5f95V8etvL18lvjJcXV0JDQ117MM+atQoJkyYgItL1U5kWLRoEc8++yx2u53S0lIee+wxxowZc9HzZ8+eTVZWFlOnTq3SOM5UWFhIdHQ027ZtO+t5w8PDmT59OpmZmcyYMQM3Nzd8fHyYNWsWbdq0Oa+ft956ixkzZmCxWAgNDeWf//znJfeSr4xdu3bxxz/+kblz517yvDp16lBcXHxee3XdLk9F50RERERuDgePH+TTTZ8yJ3cOO4t34uvlS3xQPPFB8fjU9nF2eOIEKjrnBF5eXmRnZ/Pzzz+zZMkSFi9ezIsvvlil9ygtLWX06NEsXLiQNWvWsHr1avr161el9zhXeXn5Zc9p27YtrVu35ptvvnG05eTkUFRURI8ePYiIiCArK4u1a9cSHx/Pk08+eV4fO3fu5J133iErK4v169dTXl6OzWa75vhbtGhx2WTdGSrzXkVERETk5tfQsyEPdX6Iz4Z9xtQ7phLYKJB317zLwLkDmZgxkaxfslSkTi5ICft14uvry/Tp05k6dSrGGMrLy3niiSfo3r07YWFhvP/++45z33jjDUf7888/D1SMWIeEhDBq1CjCwsKIj4+npKSEoqIiysrKaNy4MQAeHh4EBwcDFSPBY8eOpU+fPgQFBbFo0SLHPXbt2kVsbCyBgYFnJctpaWn06tWLrl27kpCQ4Bhhbtu2LS+99BLR0dGkpqZSUFBAbGws3bp1o0+fPuTk5Jz3zElJSWcl2DabjaSkJAD69+9P7dq1AYiMjGTHjh0XfG9lZWUcO3aMsrIySkpKaNGixXnn9OvXj6eeeooePXoQFBTk+JLgYu+4sLCQzp07A1BSUsLIkSMJCwsjMTGRnj17cuZsjb/85S906dKFyMhI9uzZ42hPT08/770eP36c3/72t4SGhhIREcGyZcuAihkNjzzyiOPaoUOH8vXXXwMVo/jPPfccPXv25Ntvv+Xpp5+mY8eOhIWFMXHixAu+ExERERGpGVxdXOnbui/T7pzGZ8M+4786/Bcrd63kt1/+lhELRzAndw4lpSXODlOqkZq5hv3zp+GXdZc/75e1Fb+enhp/Kc1CYfCVrYkPCAjAbrezd+9e5s+fT/369fnhhx84ceIEUVFRDBw4kPz8fPLz81m1ahXGGOLi4li+fDl+fn7k5uYyc+ZMoqKieOihh3j33XeZOHEicXFxtGnThpiYGIYOHUpSUpJjGnphYSEZGRkUFBTQv39/Nm3aBEB2djarV692JPiPPvooXl5eTJo0ifT0dLy9vXnttdd48803ee655wDw9PQkMzMTgJiYGKZNm0ZgYCDff/8948eP56uvvjrreUeOHElERARTpkzBzc2NlJQUUlNTz3svM2fOZPDgwee1t2zZkokTJ+Ln54eXlxcDBw5k4MCBF3y3ZWVlrFq1yjGLIT09nZkzZ17wHVssFsd17777Lg0bNmTt2rWsX7+e8PBwx7GjR48SGRnJK6+8wpNPPsn//u//8t///d8Xfa//+Mc/AFi3bh05OTkMHDiQvLy8S/5MHD16lM6dO/PSSy/x66+/8rvf/Y6cnBwsFguHDh265LUiIiIiUnP41fPjie5P8EjEIyzevBhbro2Xv3uZt358i3va30NicCL+9f2dHaY4Wc1M2KuR01Nb0tLSWLt2rWNq9uHDh8nPzyctLY20tDQiIiIAKC4uJj8/Hz8/P1q3bk1UVBQA999/P++88w4TJ05kxowZrFu3jvT0dCZPnsySJUuYPXs2UJE0u7i4EBgYSEBAgGMkPCYmhvr16wPQsWNHtm7dyqFDh9iwYYPjHidPnqRXr16O2BMTEx0xrVy5koSEBMexEydOnPeszZo1o1OnTixdupSmTZvi7u7uGNk+7cMPPyQrK4uMjIzzrj948CDz589ny5YtNGjQgISEBD788EPuv//+884dPnw4AN26daOwsPCS7zgoKMhxXWZmJo899hgAnTt3JiwszHGsVq1aDB061NHvkiVLHMcu9F4zMzN59NFHAQgJCaFNmzaXTdhdXV0ZMWIEAPXq1cPT05OHH36Yu+66y3FvEREREbl1eLl5MSJoBMMDh7Nm3xqSc5JJyU3ho40fEdk8EmuIlb6t+uLmotTtVnTDftctFkswkHJGUwDwHNAA+D2w71T7n40xi6/pZpUdCa/ionPn2rx5M66urvj6+mKMYcqUKQwaNOisc7788kueeeaZ84rGFRYWnjUyDJz1OTQ0lNDQUB544AH8/f0dCfvFrvHw8HC0ubq6UlZWhjGGAQMGkJycfMH4vb29AbDb7TRo0IDs7OzLPvPpafFNmzZ1TIc/LT09nVdeeYWMjIyz4jnzuL+/Pz4+FYU3hg8fzsqVKy+YsJ++/vSzABd9x6cT+tPnXIy7u7vjfZ3ZL1z4vV6sLzc3N+x2u+Pz8ePHHf/t6emJq6ur47xVq1axdOlSbDYbU6dOPW/WgoiIiIjcGiwWC+G+4YT7hvPEsSf4JP8T5uTOYcKyCTT3bs7I4JEMDxxOI89Gzg5VbqAbtobdGJNrjAk3xoQD3YAS4NNTh986feyak/VqYt++fYwdO5ZHHnkEi8XCoEGDeO+99ygtLQUgLy+Po0ePMmjQIGbNmuVYO75z50727t0LwLZt2/j2228BSE5OJjo6muLiYsd6aKiY6n5mtfXU1FTsdjsFBQVs3rzZsb79QiIjI1mxYoVj2nxJSckFR4jr1auHv7+/Y3q7MYY1a9ZcsM8RI0awePFiUlJSsFqtjvbVq1czZswYFixYgK+v7wWv9fPz47vvvqOkpARjDEuXLr2iKugXe8dnio6OZs6cOQBs2LCBdesqsXSCC7/X22+/nY8++shxr23bthEcHEzbtm3Jzs7Gbrezfft2Vq1adcE+i4uLOXz4MEOGDOHtt9+u1BciIiIiIlLzNfFqwuiw0Xwx4gve6vcWfnX9+PtPf+fO1Dt55ptnWLNvjYrU3SKcNa8iBigwxmw9d+TyZnbs2DHCw8Md27o98MADPP744wA8/PDDFBYW0rVrV4wx+Pj4MG/ePAYOHMjGjRsdU9Hr1KnDhx9+iKurKx06dOCDDz5gzJgxBAYGMm7cOMrLy3n99dcZM2YMXl5eeHt7O0bXAYKDg+nbty979uxh2rRpl9wSzcfHh9mzZ5OUlOSY4j5p0qSzppCf9tFHHzFu3DgmTZpEaWkpVquVLl26nHdegwYNHAXb/P3/s+bmiSeeoLi42DGt3s/PjwULFpx1bc+ePYmPj6dr1664ubkRERHB6NGjK/n2L/6OzzR+/HhHIb+IiAjCwsIcSwUu5ULvdfz48YwdO5bQ0FDc3NyYPXs2Hh4eREVF4e/vT2hoKJ07d6Zr164X7LOoqIh77rmH48ePY4zhrbfeqvSzioiIiEjN5+bixp1t7uTONney+dBmbLk2FhQsYNHmRXRo1IGkkCQG+w/G0+3atkGW6ssp+7BbLJZZwE/GmKkWi+UF4EHgCJAF/D9jzMFLXV9d92GvSoWFhQwdOpT169dX+prqul94dVJeXk5paSmenp4UFBQQExNDXl4etWrVcnZoV0z7sIuIiIjceo6WHmVRwSJsuTY2HdpEfY/6DGs/jJHBI2ldt7Wzw5NKquw+7Dd8hN1isdQC4oBnTjW9B7wMmFO//g/w0AWuGw2MhorR2cta9jfIqORa9hcuP8JK36eh/zOXP0+qtZKSEvr3709paSnGGN57772bMlkXERERkVuTt7s3iSGJjAweSdaeLGw5Nv614V988PMHRLeMxhpiJbplNC4W7eBdE9zwEXaLxXIP8AdjzHn7dVkslrbAImNM53OPnanKRthFbmL6mRcRERERgD1H9zA3fy5z8+ay/9h+WtVpRWJwIsMCh1HfoxKDk3LDVXaE3RlfuyQBjrLkFoul+RnHhgGVnwMuIiIiIiJyi2vq3ZQ/hP+BtBFpvHH7G/jW9uV/fvwfYlJjeG7Fc2w4sMHZIcpVuqFT4i0WS21gAHDmHmavWyyWcCqmxBeec0xEREREREQqwd3VnVj/WGL9Y8n9NZeU3BQWbV7Ep5s+JcwnDGuwlUFtB1HLVUtCbxZOKTp3rTQlXkQ/8yIiIiJyeUUni1hQsABbjo3CI4U08mzE8MDhjAwaSfM6zS/fgVwX1bbo3I3ybva7vLfmvSrrb1yXcYwPH19l/YmIiIiIiFxvdWvV5b4O95EUksR3u7/DlmNj1vpZzFo/i76t+pIUkkRk80hq0nbbNcktPcL+2y9+C8A/Y/9ZJXG5uroSGhrq2Id91KhRTJgwAReXqi0VsGjRIp599lnsdjulpaU89thjjBlz8ZUEs2fPJisri6lTp1ZpHGcqLCwkOjqabdu2nfW84eHhTJ8+nczMTGbMmIGbmxs+Pj7MmjWLNm3anNXH9u3b+c1vfsMvv/yCi4sLo0eP5rHHHgPg119/JTExkcLCQtq2bcucOXNo2LDhNcf98MMP8/jjj9OxY8eLnnOx7fK+/vprJk+ezKJFi645jquhEXYRERERuRq7ineRmpfKx3kfc/DEQdrWa4s1xEpcuzjq1qrr7PBuCdW56FyN5eXlRXZ2Nj///DNLlixh8eLFvPjii1V6j9LSUkaPHs3ChQtZs2YNq1evpl+/flV6j3OVl5df9py2bdvSunVrvvnmG0dbTk4ORUVF9OjRg4iICLKysli7di3x8fE8+eST5/Xh5ubG//zP/7Bx40a+++47/vGPf7BhQ0WBjFdffZWYmBjy8/OJiYnh1VcruWXfZcyYMeOSybozlJWVOTsEEREREanBWtRpwWNdHyM9IZ2/Rv+VerXq8eqqV4lJjeGlb18i72Ces0OUU5SwXye+vr5Mnz6dqVOnYoyhvLycJ554gu7duxMWFsb777/vOPeNN95wtD///PNAxYh1SEgIo0aNIiwsjPj4eEpKSigqKqKsrIzGjRsD4OHhQXBwMFAxEjx27Fj69OlDUFDQWSO/u3btIjY2lsDAwLOS5bS0NHr16kXXrl1JSEiguLgYqEjAX3rpJaKjo0lNTaWgoIDY2Fi6detGnz59yMnJOe+Zk5KSsNlsjs82m42kpCQA+vfvT+3atQGIjIxkx44d513fvHlzunbtCkDdunXp0KEDO3fuBGD+/PmMGjUKgFGjRjFv3rzzrp89ezbDhw+/oufs168fp2drzJw5k6CgIPr168fvf/97HnnkEcf1y5cvp3fv3gQEBDB37lxH+5EjRxg2bBgdO3Zk7Nix2O12AJKTkwkNDaVz58489dRTjvPr1Knj+O+5c+fy4IMPAhW/d48//jj9+/fnqaeeIiMjg/DwcMLDw4mIiKCoqOi85xURERERuRa1XGtxd7u7+eiuj7ANtTGo7SAWFCxgxIIRjPp8FF8UfkGpvdTZYd7SauQa9tdWvUbOr+cnlOc6fc7pqfGXEtIohKd6PHXZ884UEBCA3W5n7969zJ8/n/r16/PDDz9w4sQJoqKiGDhwIPn5+eTn57Nq1SqMMcTFxbF8+XL8/PzIzc1l5syZREVF8dBDD/Huu+8yceJE4uLiaNOmDTExMQwdOpSkpCTHNPTCwkIyMjIoKCigf//+bNq0CYDs7GxWr17tSPAfffRRvLy8mDRpEunp6Xh7e/Paa6/x5ptv8txzzwHg6elJZmYmADExMUybNo3AwEC+//57xo8fz1dffXXW844cOZKIiAimTJmCm5sbKSkppKamnvdeZs6cyeDBgy/57goLC1m9ejU9e/YEYM+ePTRvXlEUo3nz5uzdu/eC113Nc0LFFxovv/wyP/30E3Xr1uWOO+6gS5cujuO7d+8mMzOTnJwc4uLiHNPjV61axYYNG2jTpg2xsbF88skn9O7dm6eeeooff/yRhg0bMnDgQObNm8e99957yWfOy8sjPT0dV1dX7r77bv7xj38QFRVFcXExnp6el7xWRERERORadGrciZejXub/dft/zNs0j5TcFJ7IeAIfLx/ig+KJD4rHt7avs8O85dTIhL06OV0jIC0tjbVr1zpGZw8fPkx+fj5paWmkpaUREREBQHFxMfn5+fj5+dG6dWuioqIAuP/++3nnnXeYOHEiM2bMYN26daSnpzN58mSWLFnC7NmzgYqk2cXFhcDAQAICAhwj4TExMdSvXx+Ajh07snXrVg4dOsSGDRsc9zh58iS9evVyxJ6YmOiIaeXKlSQkJDiOnThx4rxnbdasGZ06dWLp0qU0bdoUd3d3OnfufNY5H374IVlZWWRkZFz0nRUXFzNixAjefvtt6tWrV8k3zVU/J1Qk3n379qVRo0YAJCQkkJf3n6lA9957Ly4uLnTs2JE9e/Y42nv06EFAQABQMcMgMzMTd3d3+vXrh4+PDwD33Xcfy5cvv2zCnpCQgKurKwBRUVE8/vjj3HfffQwfPpxWrVpd0XsQEREREbkaDTwb8GDnB/lNp9+QuTOT5Jxkpq2Zxv+u/V/u8LsDa4iV25repiJ1N0iNTNgrOxJe1UXnzrV582ZcXV3x9fXFGMOUKVMYNGjQWed8+eWXPPPMM+cVjSssLDzvD8GZn0NDQwkNDeWBBx7A39/fkbBf7BoPDw9Hm6urK2VlZRhjGDBgAMnJyReM39vbGwC73U6DBg3Izs6+7DOfnhbftGlTx3T409LT03nllVfIyMg4K54zlZaWMmLECEeielrTpk3ZvXs3zZs3Z/fu3fj6Xvjbvat5TvjPFysXc2a/Z557ofd9qb7OPP/48eNnHTv9vgGefvpp7rrrLhYvXkxkZCTp6emEhIRcMkYRERERkariYnHh9la3c3ur29l+ZDspuSl8uulT0ram0b5Be6zBVu5udze13Ws7O9QaTWvYr5N9+/YxduxYHnnkESwWC4MGDeK9996jtLRiDUheXh5Hjx5l0KBBzJo1y7GmeufOnY7p3tu2bePbb78FKtZER0dHU1xczNdff+24T3Z29lnV1lNTU7Hb7RQUFLB582bH+vYLiYyMZMWKFY5p8yUlJWeNKp9Wr149/P39HdPbjTGsWbPmgn2OGDGCxYsXk5KSgtVqdbSvXr2aMWPGsGDBgosm28YYfve739GhQwcef/zxs47FxcXxwQcfAPDBBx9wzz33XPS5ruY5e/ToQUZGBgcPHqSsrIyPP/64Un2vWrWKLVu2YLfbSUlJITo6mp49e5KRkcH+/fspLy8nOTmZvn37AhVfPGzcuBG73c6nn3560X4LCgoIDQ3lqaee4rbbbrtgzQARERERkRuhdb3WTOw+kfSEdF7q/RLuLu5M+n4Sd6Tewd++/xubD292dog1Vo0cYXeWY8eOER4e7tjW7YEHHnAkng8//DCFhYV07doVYww+Pj7MmzePgQMHsnHjRscU7Tp16vDhhx/i6upKhw4d+OCDDxgzZgyBgYGMGzeO8vJyXn/9dcaMGYOXlxfe3t6O0XWA4OBg+vbty549e5g2bdol1z77+Pgwe/ZskpKSHFPcJ02aRFBQ0HnnfvTRR4wbN45JkyZRWlqK1Wo9a433aQ0aNCAyMpI9e/bg7+/vaH/iiScoLi52TKv38/NjwYIFZ127YsUK/vWvfxEaGkp4eDgAf/3rXxkyZAhPP/00I0eOZObMmfj5+V1wbfy1PGfLli3585//TM+ePWnRogUdO3Z0TK2/lF69evH000+zbt06br/9doYNG4aLiwt/+9vf6N+/P8YYhgwZ4viC4dVXX2Xo0KG0bt2azp07O76oOdfbb7/NsmXLcHV1pWPHjpdd8y8iIiIicr15uXkxLHAY97a/l7X715Kck0xqXir/zvk3PZv3JCk4ib6t++LmojSzqmgfdq7flPhrUVhYyNChQ1m/fn2lr7nYfuFSOcXFxdSpU4eysjKGDRvGQw89xLBhw5wd1kVpH3YRERERcbYDxw7wSf4nzMmbwy9Hf6GZdzMSghIYETiCxl6NnR1etVXZfdhr7Fcf72a/y3tr3qvUuaEfhF72nHFdxjE+fPy1hiXV2AsvvEB6ejrHjx9n4MCBly0SJyIiIiJyq2vs1Zjfh/2e33b+LRk7MrDl2JiyegrvrXmPQW0HYQ220sWni4rUXaVbeoRd5Gamn3kRERERqY42H95MSk4KCwoWUFxaTIdGHbCGWBnsPxgvNy9nh1ctVHaEXUXnREREREREpMoE1A/gmZ7PsDRhKc9GPkupvZTnVz7Pnal38sYPb7DtyDZnh3jTqLFT4kVERERERMR5arvXZmTwSBKCEvhxz4/Ycm38e+O/+b8N/0d0y2iSQpKIahGFq4urs0OttpSwi4iIiIiIyHVjsVi4rdlt3NbsNvaW7GVu3lzm5s3lD0v/QMs6LUkMTmRY+2E08Gzg7FCrnRq7hn3flKns/8c/quyeTf7wB3wefaTK+hO5VlrDLiIiIiI3q1J7KUu3LcWWY+PHPT/i4epBbNtYkkKS6NSkk7PDu+4qu4a9xibslbH1gd8A0OZf/1clcbm6uhIaGurYh33UqFFMmDABF5eqLRWwaNEinn32Wex2O6WlpTz22GOMGTPmoufPnj2brKwspk6dWqVxnKmwsJDo6Gi2bdt21vOGh4czffp0MjMzmTFjBm5ubvj4+DBr1izatGlzwb7Ky8u57bbbaNmyJYsWLQJgy5YtWK1Wfv31V7p27cq//vUvatWqdc1xDxkyhH//+980aHDxb/P69evH5MmTue22s/883Yj3eilK2EVERESkJsg7mEdKTgoLNy/kWNkxwpqEYQ2xMrDtQDxcPZwd3nWhonNO4OXlRXZ2Nj///DNLlixh8eLFvPjii1V6j9LSUkaPHs3ChQtZs2YNq1evpl+/flV6j3OVl5df9py2bdvSunVrvvnmG0dbTk4ORUVF9OjRg4iICLKysli7di3x8fE8+eSTF+3r73//+3mJ6FNPPcWf/vQn8vPzadiwITNnzrz6BzrD4sWLL5msO0NZWZmzQxARERERuWGCGgbxbK9nWZqwlKd7PM2Rk0f4c+afGZA6gLd/fJtdxbucHaLTKGG/Tnx9fZk+fTpTp07FGEN5eTlPPPEE3bt3JywsjPfff99x7htvvOFof/7554GKEeuQkBBGjRpFWFgY8fHxlJSUUFRURFlZGY0bNwbAw8OD4OBgAB588EHGjh1Lnz59CAoKcoxOA+zatYvY2FgCAwPPSpbT0tLo1asXXbt2JSEhgeLiYqAiAX/ppZeIjo4mNTWVgoICYmNj6datG3369CEnJ+e8Z05KSsJmszk+22w2kpKSAOjfvz+1a9cGIDIykh07dlzwve3YsYPPPvuMhx9+2NFmjOGrr74iPj4egFGjRjFv3rzzrn3hhRd46KGH6NevHwEBAbzzzjuOYx9++CE9evQgPDycMWPGOL6EaNu2Lfv37wfg5ZdfJiQkhAEDBpCUlMTkyZMd16emptKjRw+CgoLO+lJi+/btxMbGEhwcfNaXM2+++SadO3emc+fOvP3220DF72nnzp0d50yePJkXXngBqBjF//Of/0zfvn35+9//TmpqKp07d6ZLly7cfvvtF3xXIiIiIiI1Sd1adbmvw30suHd+69JAAAAQkUlEQVQB0wdMJ8I3gn/+/E8GfzKYR796lJU7V2I3dmeHeUPVyKJzv/z1r5zYeH5Cea7jp5LO01PjL8WjQwjN/vznK4ojICAAu93O3r17mT9/PvXr1+eHH37gxIkTREVFMXDgQPLz88nPz2fVqlUYY4iLi2P58uX4+fmRm5vLzJkziYqK4qGHHuLdd99l4sSJxMXF0aZNG2JiYhg6dChJSUmOaeiFhYVkZGRQUFBA//792bRpEwDZ2dmsXr3akeA/+uijeHl5MWnSJNLT0/H29ua1117jzTff5LnnngPA09OTzMxMAGJiYpg2bRqBgYF8//33jB8/nq+++uqs5x05ciQRERFMmTIFNzc3UlJSSE1NPe+9zJw5k8GDB1/wnU2YMIHXX3+doqIiR9uBAwdo0KABbm4VP66tWrVi586dF7w+JyeHZcuWUVRURHBwMOPGjWPTpk2kpKSwYsUK3N3dGT9+PB999BG/+c1/ft+zsrL4+OOPWb16NWVlZXTt2pVu3bo5jpeVlbFq1SrHrIn09HQAVq1axfr166lduzbdu3fnrrvuwmKx8M9//pPvv/8eYww9e/akb9++NGzY8BI/LXDo0CEyMjIACA0N5csvv6Rly5YcOnTokteJiIiIiNQkFouFXi160atFL3YX7yY1L5WP8z/m6+1f07ZeWxKDE4lrH0e9WvWcHep1VyMT9urkdI2AtLQ01q5dy9y5cwE4fPgw+fn5pKWlkZaWRkREBADFxcXk5+fj5+dH69atiYqKAuD+++/nnXfeYeLEicyYMYN169aRnp7O5MmTWbJkCbNnzwYqkmYXFxcCAwMJCAhwjITHxMRQv359ADp27MjWrVs5dOgQGzZscNzj5MmT9OrVyxF7YmKiI6aVK1eSkJDgOHbixInznrVZs2Z06tSJpUuX0rRpU9zd3c8aUYaKke6srCxHYnqmRYsW4evrS7du3fj666/Pe4dnslgsF3zfd911Fx4eHnh4eODr68uePXtYunQpP/74I927dwfg2LFj+Pr6nnVdZmYm99xzD15eXgDcfffdZx0fPnw4AN26daOwsNDRPmDAAMdsh+HDh5OZmYnFYmHYsGF4e3s72r/55hvi4uIuGPNpp983QFRUFA8++CAjR4503FtERERE5FbTvE5z/tj1j4ztMpa0rWnYcmy89sNrvLP6He4KuAtrsJXgRsHODvO6qZEJe2VHwqu66Ny5Nm/ejKurK76+vhhjmDJlCoMGDTrrnC+//JJnnnnmvKJxhYWF5yWlZ34ODQ0lNDSUBx54AH9/f0fCfrFrPDz+U6zB9f+3d//BWVXpAce/j/wQV6rslixVUcRWRatLzKLgiqCgYLsp2NrWaGWAhRXpstsdO9jpjnVXi0txdrrdtlSWtWoGXGBlRgxC3YGx6pSxbP0RthYsgg0YO0NiWH+EEArk9I/3TUxCXokSkvd9/X5m7pB7z7n3fc5LZk6ee849t18/jhw5QkqJG2+8kVWrVnUZf2vC2dLSwpAhQ6iurj5um1unxQ8bNqxtOnyrzZs38+CDD/LCCy90iKfVli1bqKqqYuPGjTQ3N/PBBx9wxx13sGLFCt577z2OHDlC//79qa2t5eyzz+7y83O1c+bMmSxevDhn3MdbfLH1uq3XbNXV953rWv3796el5aMpPM3NzR3KW79vgGXLlrF161Y2bNhAaWkp1dXVbTcGJEmSpM+agf0GUn5BOeUXlLO9YTur31jN+t3rWbtzLWVfLKNiVAU3nHcDA/oN6OtQe5TPsJ8k9fX13HXXXSxYsICIYOrUqTz88MMcPnwYgJ07d3LgwAGmTp3Ko48+2vbs+DvvvENdXR0Ae/fu5aWXXgJg1apVjB8/nsbGxg6jz9XV1R1WW3/yySdpaWlh9+7dvPXWW23Pt3dl3LhxbNmypW3afFNTEzt37jym3hlnnMHIkSPbprenlNi2bVuX17zlllvYuHEja9asoaKiou34a6+9xrx586iqqjpmdLvV4sWLqa2tpaamhtWrVzNp0iRWrlxJRHD99de3zU6orKxk+vTpOdvV2eTJk1m7dm3b97p//3727NnToc748eNZv349zc3NNDY2smHDhm5de9OmTezfv5+DBw+ybt06rrnmGiZMmMC6detoamriwIEDPPXUU1x77bUMGzaMuro6GhoaOHToUIc1BjrbvXs3Y8eO5YEHHmDo0KG8/fbb3W6vJEmSVCxmPzubyysv77Dd+sytPLXrKQ4dzcz6fbXuVe558R7KVpYdU7fzNvvZ2X3cok+mKEfY+8rBgwcpLS1te63bjBkzuPvuuwGYO3cuNTU1lJWVkVKipKSEdevWMWXKFHbs2NE2FX3w4MGsXLmSfv36cckll1BZWcm8efO48MILmT9/PkePHuWhhx5i3rx5nHbaaZx++ulto+sAF198MRMnTmTfvn0sW7aMQYMG5Yy3pKSExx9/nNtuu61tivuiRYu46KKLjqn7xBNPMH/+fBYtWsThw4epqKhg9OjRx9QbMmQI48aNY9++fYwcObLt+MKFC2lsbGybVn/eeedRVVXV7e92yZIlVFRUcO+993LFFVcwZ86cbp976aWXsmjRIqZMmUJLSwsDBgxg6dKlHW50XHnllUybNo3Ro0czYsQIxowZ0/YIwccZP348M2bMYNeuXdx+++1tr36bNWsWV111FZD5v2995OG+++5j7NixjBw5klGjRuW87sKFC3nzzTdJKTF58uQuv2tJkiSp2D1202PHrdOSWnhl3yuMGTYm56Ozhcr3sHPypsSfiJqaGsrLy3n99de7fc6sWbMoLy9vW01dn0xjYyODBw+mqamJCRMmsHz5csrKyvo6rJx8D7skSZJUmLr7HvaiHWGv/4d/5N2lS7tVd8eo4yc9Q7/xDUq+ueBEw1Ieu/POO9m+fTvNzc3MnDkzr5N1SZIkScXvMz3CLhUyf+clSZKkwtTdEXYXnZMkSZIkKQ8VVcJeiLMFpE/D33VJkiSp+BVNwj5o0CAaGhpMZFT0Uko0NDR87BsAJEmSJBW+oll0bvjw4dTW1lJfX9/XoUgn3aBBgxg+fHhfhyFJkiTpJCqahH3AgAEd3vstSZIkSVIhK5op8ZIkSZIkFRMTdkmSJEmS8pAJuyRJkiRJeSgKcVX1iKgH9vR1HN00FHi3r4OQJOkTsO+SJBWaQuu7RqSUSo5XqSAT9kISES+nlMb0dRySJHWXfZckqdAUa9/llHhJkiRJkvKQCbskSZIkSXnIhP3kW97XAUiS9AnZd0mSCk1R9l0+wy5JkiRJUh5yhF2SJEmSpDxkwt4DImJQRPwiIrZFxH9FxP1d1Dk1ItZExK6I2BoR5/d+pJIkZXSz75oVEfURUZ3d5vZFrJIktRcR/SLitYh4pouyosq7TNh7xiFgUkppNFAK3BQR4zrVmQP8KqX0W8APgSW9HKMkSe11p+8CWJNSKs1uj/RuiJIkdenPgB05yooq7zJh7wEpozG7OyC7dV4cYDpQmf15LTA5IqKXQpQkqYNu9l2SJOWViBgOfBXIdRO5qPIuE/Yekp2WUQ3UAZtSSls7VTkHeBsgpXQEeB/49d6NUpKkj3Sj7wK4JSJ+GRFrI+LcXg5RkqTO/g64B2jJUV5UeZcJew9JKR1NKZUCw4GrIuKyTlW6uqvjSIYkqc90o+9aD5yfUvoSsJmPRiwkSep1EVEO1KWUXvm4al0cK9i8y4S9h6WU3gOeB27qVFQLnAsQEf2BM4H9vRqcJEldyNV3pZQaUkqHsrs/Ab7cy6FJktTeNcC0iKgBVgOTImJlpzpFlXeZsPeAiCiJiCHZn08DbgDe6FStCpiZ/fkPgedSSgV7p0eSVNi603dFxFntdqeRe4EfSZJOupTSX6aUhqeUzgcqyORUd3SqVlR5V/++DqBInAVURkQ/MjdBfpZSeiYiHgBeTilVAf8MrIiIXWTu8FT0XbiSJHWr7/pWREwDjpDpu2b1WbSSJOVQzHlXFPDNBkmSJEmSipZT4iVJkiRJykMm7JIkSZIk5SETdkmSJEmS8pAJuyRJkiRJeciEXZIkSZKkPGTCLklSnoiI6yIiRcTQvo7lZPkstFGSpJ7ia90kScoTETEQ+AKwLxVpB/1ZaKMkST3FhF2SJEmSpDzklHhJknpRREyIiH+PiMaIeD8itkbEZdmyY6aLR8TXImJvRDRFxPqI+NOISO3KvxcRr0fEzIioyV73sYgYmK37dkQ0RMTfRsQp7c67IyL+IyI+jIi6iHgyIs45TuzPR8Q/RcT3I+Ld7Hk/6HTdz0dEZUT8KiIORsTmiPjtduUd2hgRZ0bEiuy1miPirYj4drv6Z0bE8mz5hxHxQkSMOdH/B0mSCoEJuyRJvSQi+gNPA/8GjAbGAj8CjuaofzXwCLAUKAWqgPu7qHo+MB0oB24B/ij7OVcCU4C5wDeB3293zkDgu9k4yoGhwKpuNONPgCPAV4AFwLeBW9uVP55t13TgKqAJeDYiTstxvUXA5dkYRgFfA97Jtj+ADcA52fIrgBeB5yLirG7EKklSQXNKvCRJvSQivgA0ANellF7oovw64F+BkpTSuxGxCvh8SummdnWWA19PKUV2/3vAXwC/kVJ6P3tsLTAROCel9H/ZY88Dr6eUFuSIbRSwAzg3pVSbo87zwKkppavbHdsE7EkpzY2IC4GdwMSU0ovZ8jOBvcCfp5Qe6aKNVUBDSml2F583icxNipKU0sF2x6uBn6aUHuoqTkmSioUj7JIk9ZKU0n4yI9A/j4gNEXF3RJz7MaeMAn7R6djWLurtbU3Ws/YBO1uT9XbHvti6ExFlEfF0ROyJiA+Bl7NF5x2nGb/stP+/7a57CdACvNRamI3rP4FLc1zvYeCPI2Jbdnr9xHZlXwY+B9Rnp/o3RkQjcBnwm8eJU5KkgmfCLklSL8qOJI8lM7V7GrAzIqbmqB5Ad6bCHe78MTmOnQIQEacDPyczXX0GmanzraP4Az/FZ7X+PREfc16X7Ugp/QswAvgBmWn5GyLisWzxKWRuNJR22kYBf3WcOCVJKngm7JIk9bKU0raU0pKU0nXA88DMHFV3kHkOvL3O+5/GKDLJ8XdSSi+mlN6g3ej7CdhO5m+L9lPmzyDzjPr2XCellN5NKa1IKc0C5gAzI+JU4FVgGNCSUtrVaavrgXglScprJuySJPWSiBgZEX8TEV+JiBERcT3wJXIns38PTImIhRFxYUTMoePCcZ/WXuAQsCAiLoiIrwJ/faIXTSm9SWaxux9HxLURcTmwEvgA+GlX50TEAxFxc7Z9lwB/ALyVUjoEbAa2AE9HxO9kv7+rI+L+iLj2ROOVJCnfmbBLktR7moCLgCfJLM5WCTwBLOmqckrpJeDrwLfIPDt+c7Zu84kEkVKqJzOqfzOZmwXfBe4+kWu2M5vMc/dV2X8/B9zUftG4Tg4BDwLbyCTnvwb8XjbOBPwu8BzwE+C/gZ8BF5N5dl6SpKLmKvGSJBWQiPghcENK6fK+jkWSJJ1c/fs6AEmSlFtELAQ2AY3ADcBdwHf6NChJktQrHGGXJCmPRcQa4DrgTOB/gB8DP0p24JIkFT0TdkmSJEmS8pCLzkmSJEmSlIdM2CVJkiRJykMm7JIkSZIk5SETdkmSJEmS8pAJuyRJkiRJeciEXZIkSZKkPPT/J4NGZ1+Tl9EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1224x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "batch_size = 8\n",
    "batch_time_V1 = [1.49936, 1.48389, 1.53598, 1.55628, 1.474281]\n",
    "# batch_time_V2_50 = [3.4973, 3.5826, 3.36076]\n",
    "batch_time_V2_40 = [2.95925, 2.86425, 2.87896]\n",
    "batch_time_V2_20 = [2.09028, 2.04751, 1.96164, 2.00741, 1.94831, 1.93147716, 1.99608524]\n",
    "batch_time_V2_8  = [1.47995, 1.48571, 1.48775]\n",
    "training_time = {\"V1\": \"187 1h45\",\n",
    "                 \"V1 CNN\": \"186 1h45\"\n",
    "#                  \"V2 50\": \"3h10\",\n",
    "                 \"V2 40\": \"363  3h45\",\n",
    "                 \"V2 20\": \"250  2h30\",\n",
    "                 \"V2 8\":  \"185 1h45\",}\n",
    "noise_ = [3, 3.5, 4]# [2, 3, 4, 6, 8]\n",
    "# new_graph_res = [99.17, np.nan, 94.58, 74.58, 68.33]\n",
    "new40_graph_res = [[98.33, 92.5, 91.67]]#[98.33, 96.67, 94.17], [np.nan, np.nan, 91.67]]\n",
    "new20_graph_res = [[np.nan, 91.25, 72.08]]#[95.42, 98.34, 81.67], [np.nan, 91.25, 72.08]]\n",
    "#                     [[99.58, 99.58, 75.0, 67.92, 66.25], [99.58, np.nan,72.5,66.67,66.25], #[np.nan, np.nan, 71.25, 62.08]\n",
    "#                    [99.58, np.nan,81.67,54.17,57.5], [100, np.nan, 80.42, 74.58,68.34]]\n",
    "new8_graph_res = [[97.5, 87.08, 97.08]]#[97.5, 90, 73.34], [94.17, 87.08, 81.67]]\n",
    "#                 [[99.17, 90.0, 90.42, 62.92, 64.12],[99.58, 98.75, 83.33, 73.75, 63.75], [100, np.nan, 68.34, 69.17, 57.08], \n",
    "#                   [100, np.nan, 82.92, 62.92, 57.5]]\n",
    "old_graph_res = [[95.83, 83.75, np.nan]]#[95.83, 91.25, 71.25], [np.nan, 83.75, np.nan]]\n",
    "#                 [[99.17, 98.34, 94.58, 61.25, 57.5], [np.nan, np.nan,79.17,62.08,62.08], # ,[99.17, 93.75, 70.42, 70.42]\n",
    "#                  [100, np.nan, 83.75, 57.5, 61.25], [100, np.nan, 94.58, 75.0, 60.0], [100, np.nan, 72.67, 62.08, np.nan]]\n",
    "plt.errorbar(noise_, np.nanmean(old_graph_res, axis=0), yerr=np.nanstd(old_graph_res, axis=0), capsize=10,label='DeepSphere V1')\n",
    "plt.errorbar(noise_, np.nanmean(new8_graph_res, axis=0), yerr=np.nanstd(new8_graph_res, axis=0), capsize=10, label='DeepSphere V2 8 neighbours')\n",
    "plt.errorbar(noise_, np.nanmean(new20_graph_res, axis=0), yerr=np.nanstd(new20_graph_res, axis=0), capsize=10, label='DeepSphere V2 20 neighbours')\n",
    "plt.errorbar(noise_, np.nanmean(new40_graph_res, axis=0), yerr=np.nanstd(new40_graph_res, axis=0), capsize=10, label='DeepSphere V2 40 neighbours')\n",
    "# plt.errorbar(noise_, new_graph_res, yerr=[0, 0, 0, 0, 0], label='DeepSphere V2 50 neighbours')\n",
    "plt.legend()\n",
    "plt.xlabel('sigma noise', fontsize=14)\n",
    "plt.ylabel('Accuracy (%)', fontsize=14)\n",
    "plt.title('Performance on test set with order 1 for cosmo experiment', fontsize=14)\n",
    "# plt.xlim([2.8,4.2])\n",
    "plt.xticks([3,3.5,4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2 Solve the problem using histogram features and an SVM classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Features compuation and dataset creation\n",
    "The following function prepare the features for the SVM classifier.\n",
    "1. It splits the training data into a training and a validation set.\n",
    "2. It augments the training set by adding different realization of random noise to the sample\n",
    "3. It computes the histogram features for the training, validation and testing set.\n",
    "4. It normalizes the features in order for them to have a mean of 0 and a variance of 1.\n",
    "\n",
    "The features are computed using the function `histogram` of `experiment_helper.py`.\n",
    "\n",
    "We use 10 different noise realization by setting `augmentation=10` in order to increase the number of training sample."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ret = experiment_helper.data_preprossing(x_raw_train, labels_raw_train, x_raw_test, sigma_noise, feature_type='histogram', augmentation=10)\n",
    "features_train, labels_train, features_validation, labels_validation, features_test = ret "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2  Classification using SVM\n",
    "Let us test classify our data using an SVM classifier."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "error_train, error_validation, C = experiment_helper.err_svc_linear(features_train, labels_train, features_validation, labels_validation)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "print('The validation error is {}%'.format(error_validation * 100), flush=True)\n",
    "print('The Training error is {}%'.format(error_train * 100), flush=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now compute the error on the testing set. To avoid complexity, we do a small mistake that advantage the SVM classifer: we do cross-validation on the testing set.\n",
    "\n",
    "While this is wrong, the spherical CNN still clearly outperform the SVM classifier."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_, error_test = experiment_helper.err_svc_linear_single(C, features_train, labels_train, features_test, labels_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "print('The testing error is {}%'.format(error_test * 100), flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Histogram features visualization\n",
    "\n",
    "To get a grasp of what is happening, let us plot the histogram of the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "cmin = np.min(x_raw_train)\n",
    "cmax = np.max(x_raw_train)\n",
    "bins = 100\n",
    "x = np.linspace(cmin,cmax,bins)\n",
    "\n",
    "fig, axes = plt.subplots(1, 2)\n",
    "x_hist = experiment_helper.histogram(x_raw_train, cmin, cmax)\n",
    "plot.plot_with_std(x, x_hist[labels_raw_train==0], color='b', label='class 1', ax=axes[0])\n",
    "plot.plot_with_std(x, x_hist[labels_raw_train==1], color='r', label='class 2', ax=axes[0])\n",
    "axes[0].legend()\n",
    "axes[0].set_title('Histogram - Noiselss case');\n",
    "\n",
    "if sigma_noise:\n",
    "    # Updating cmin and cmax does not really affect the features. \n",
    "    # We keep the same as in the noisless case in order to have the same x axis.\n",
    "    x_hist = experiment_helper.histogram(x_raw_train+sigma_noise*np.random.randn(*x_raw_train.shape), cmin, cmax)\n",
    "    plot.plot_with_std(x, x_hist[labels_raw_train==0], color='b', label='class 1', ax=axes[1])\n",
    "    plot.plot_with_std(x, x_hist[labels_raw_train==1], color='r', label='class 2', ax=axes[1])\n",
    "    axes[1].legend()\n",
    "    axes[1].set_title('Histogram-  Noisy case');\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These histogram are normalized in order to get the final features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ax = plot.plot_with_std(features_train[labels_train==0], color='b', label='class 1')\n",
    "ax = plot.plot_with_std(features_train[labels_train==1], color='r', label='class 2', ax=ax)\n",
    "ax.legend()\n",
    "ax.set_title('Histogram features');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ax = plot.plot_with_std(features_validation[labels_validation==0,:80], color='b', label='class 1')\n",
    "# ax = plot.plot_with_std(features_validation[labels_validation==1,:80], color='r', label='class 2', ax=ax)\n",
    "# ax.legend()\n",
    "# ax.set_title('Histogram features - Validation set');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ax = plot.plot_with_std(features_test[labels_test==0,:80], color='b', label='class 1')\n",
    "# ax = plot.plot_with_std(features_test[labels_test==1,:80], color='r', label='class 2', ax=ax)\n",
    "# ax.legend()\n",
    "# ax.set_title('Histogram features - Test set');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3 Solve the problem using PSD features and an SVM classifier\n",
    "Solving the problem with PSD features is very similar than solving it with histogram features. Hence we are not describing each step.\n",
    "\n",
    "The computation of the PSD features is actually very expensive. Since the classifier will also fail miserably, you may just want to not exectute this part of the notebook. In order to reduce the amount of PSD to be computed, we disable the dataset augementation by setting `augmentation=1`. Nevertheless, we use augmentation for the results in the paper."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ret = experiment_helper.data_preprossing(x_raw_train, labels_raw_train, x_raw_test, sigma_noise, feature_type='psd', augmentation=1)\n",
    "features_train, labels_train, features_validation, labels_validation, features_test = ret "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "error_train, error_validation, C = experiment_helper.err_svc_linear(features_train, labels_train, features_validation, labels_validation)\n",
    "print('The validation error is {}%'.format(error_validation * 100), flush=True)\n",
    "print('The Training error is {}%'.format(error_train * 100), flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_, error_test = experiment_helper.err_svc_linear_single(C, features_train, labels_train, features_test, labels_test)\n",
    "print('The testing error is {}%'.format(error_test * 100), flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 PSD features visualization\n",
    "\n",
    "To get a grasp of what is happening, let us plot the psd features. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ell = np.arange(features_train.shape[1])\n",
    "ax = plot.plot_with_std(ell, features_train[labels_train==0], color='b', label='class 1')\n",
    "ax = plot.plot_with_std(ell, features_train[labels_train==1], color='r', label='class 2', ax=ax)\n",
    "ax.legend()\n",
    "ax.set_title('PSD features');\n",
    "# plt.xscale('log')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ell = np.arange(features_train.shape[1])\n",
    "# ax = plot.plot_with_std(ell, features_validation[labels_validation==0], color='b', label='class 1')\n",
    "# ax = plot.plot_with_std(ell, features_validation[labels_validation==1], color='r', label='class 2', ax=ax)\n",
    "# ax.legend()\n",
    "# ax.set_title('PSD features - validation dataset');\n",
    "# # plt.xscale('log')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ell = np.arange(features_train.shape[1])\n",
    "# ax = plot.plot_with_std(ell, features_test[labels_test==0], color='b', label='class 1')\n",
    "# ax = plot.plot_with_std(ell, features_test[labels_test==1], color='r', label='class 2', ax=ax)\n",
    "# ax.legend()\n",
    "# ax.set_title('PSD features - testing dataset');\n",
    "# # plt.xscale('log')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4 Classification using Deep Sphere\n",
    "\n",
    "Let us now classify our data using a spherical convolutional neural network."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 Preparation of the dataset\n",
    "Let us create the datafor the spherical neural network. It is simply the raw data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Class 1 VS class 2\n",
      "  Training set: 195 / 189\n",
      "  Validation set: 45 / 51\n"
     ]
    }
   ],
   "source": [
    "ret = experiment_helper.data_preprossing(x_raw_train, labels_raw_train, x_raw_test, sigma_noise, feature_type=None)\n",
    "features_train, labels_train, features_validation, labels_validation, features_test = ret"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The spherical neural network will uses a Dataset object that need to be initialized. The object `LabeledDatasetWithNoise` will add noise to the raw data at the time of training. It will slowly increase the amount of noise during `nit` iteration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 155,
   "metadata": {},
   "outputs": [],
   "source": [
    "training = LabeledDatasetWithNoise(features_train, labels_train, end_level=sigma_noise)\n",
    "validation = LabeledDataset(features_validation, labels_validation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 Building the Network\n",
    "\n",
    "We now create our spherical neural network. We use one architecture, a fully convolutional architecture (see the exact parameters in `hyperparameters.py`), for all the problems (that is for all configurations of `order` and `sigma_noise`. A smaller `order` means more pixels per sample, that is more data for a prediction. It translates to higher accuracy as the network is more confident about its prediction (as they are averaged across spatial locations).\n",
    "\n",
    "For the paper, we selected a conservative set of parameters that were providing good results across the board. To train faster, diminish `num_epochs`, or interrupt training whenever you get bored. To reproduce all the results from the paper, the easiest is to run the `experiments_deepsphere.py` script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {},
   "outputs": [],
   "source": [
    "ntype = 'FCN'\n",
    "EXP_NAME = 'oldgraph_{}sides_{:0.1f}noise_{}order_{}sigma_{}'.format(Nside, sigma_noise, order, sigma, ntype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Cleanup before running again.\n",
    "shutil.rmtree('summaries/{}/'.format(EXP_NAME), ignore_errors=True)\n",
    "shutil.rmtree('checkpoints/{}/'.format(EXP_NAME), ignore_errors=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_params(ntrain, EXP_NAME, order, Nside, architecture=\"FCN\", verbose=True):\n",
    "    \"\"\"Parameters for the cgcnn and cnn2d defined in deepsphere/models.py\"\"\"\n",
    "\n",
    "    n_classes = 2\n",
    "\n",
    "    params = dict()\n",
    "    params['dir_name'] = EXP_NAME\n",
    "\n",
    "    # Types of layers.\n",
    "    params['conv'] = 'chebyshev5'  # Graph convolution: chebyshev5 or monomials.\n",
    "    params['pool'] = 'max'  # Pooling: max or average.\n",
    "    params['activation'] = 'relu'  # Non-linearity: relu, elu, leaky_relu, softmax, tanh, etc.\n",
    "    params['statistics'] = 'mean'  # Statistics (for invariance): None, mean, var, meanvar, hist.\n",
    "\n",
    "    # Architecture.\n",
    "    params['F'] = [16, 32, 64, 64, 64, n_classes]  # Graph convolutional layers: number of feature maps.\n",
    "    params['K'] = [5] * 6  # Polynomial orders.\n",
    "    params['batch_norm'] = [True] * 6  # Batch normalization.\n",
    "    params['M'] = []  # Fully connected layers: output dimensionalities.\n",
    "\n",
    "    # Pooling.\n",
    "    nsides = [Nside, Nside//2, Nside//4, Nside//8, Nside//16, Nside//32, Nside//32]\n",
    "    params['nsides'] = nsides\n",
    "    params['indexes'] = utils.nside2indexes(nsides, order)\n",
    "#     params['batch_norm_full'] = []\n",
    "\n",
    "    if architecture == \"CNN\":\n",
    "        # Classical convolutional neural network.\n",
    "        # Replace the last graph convolution and global average pooling by a fully connected layer.\n",
    "        # That is, change the classifier while keeping the feature extractor.\n",
    "        params['F'] = params['F'][:-1]\n",
    "        params['K'] = params['K'][:-1]\n",
    "        params['batch_norm'] = params['batch_norm'][:-1]\n",
    "        params['nsides'] = params['nsides'][:-1]\n",
    "        params['indexes'] = params['indexes'][:-1]\n",
    "        params['statistics'] = None\n",
    "        params['M'] = [n_classes]\n",
    "    elif architecture == \"FCN\":\n",
    "        pass\n",
    "    elif architecture == 'CNN-2d':\n",
    "        params['F'] = [8, 16, 32, 32, 16]\n",
    "        params['K'] = [[5, 5]] * 5\n",
    "        params['p'] = [2, 2, 2, 2, 2]\n",
    "        params['input_shape'] = [1024//order, 1024//order]\n",
    "        params['batch_norm'] = params['batch_norm'][:-1]\n",
    "        params['statistics'] = None\n",
    "        params['M'] = [n_classes]\n",
    "        del params['indexes']\n",
    "        del params['nsides']\n",
    "        del params['conv']\n",
    "\n",
    "    elif architecture == 'FCN-2d':\n",
    "        params['F'] = [8, 16, 32, 32, 16, 2]\n",
    "        params['K'] = [[5, 5]] * 6\n",
    "        params['p'] = [2, 2, 2, 2, 2, 1]\n",
    "        params['input_shape'] = [1024//order, 1024//order]\n",
    "        del params['indexes']\n",
    "        del params['nsides']\n",
    "        del params['conv']\n",
    "    else:\n",
    "        raise ValueError('Unknown architecture {}.'.format(architecture))\n",
    "\n",
    "    # Regularization (to prevent over-fitting).\n",
    "    params['regularization'] = 0  # Amount of L2 regularization over the weights (will be divided by the number of weights).\n",
    "    if '2d' in architecture:\n",
    "        params['regularization'] = 3\n",
    "    params['dropout'] = 1  # Percentage of neurons to keep.\n",
    "\n",
    "    # Training.\n",
    "    params['num_epochs'] = 80  # Number of passes through the training data.\n",
    "    params['batch_size'] = max(8 * order, 1)    # Constant quantity of information (#pixels) per step (invariant to sample size).\n",
    "\n",
    "    # Optimization: learning rate schedule and optimizer.\n",
    "    params['scheduler'] = lambda step: tf.train.exponential_decay(2e-4, step, decay_steps=1, decay_rate=0.999)\n",
    "    params['optimizer'] = lambda lr: tf.train.AdamOptimizer(lr, beta1=0.9, beta2=0.999, epsilon=1e-8)\n",
    "\n",
    "    # Number of model evaluations during training (influence training time).\n",
    "    n_evaluations = 80\n",
    "    params['eval_frequency'] = int(params['num_epochs'] * ntrain / params['batch_size'] / n_evaluations)\n",
    "\n",
    "    if verbose:\n",
    "        print('#sides: {}'.format(nsides))\n",
    "#         print('#pixels: {}'.format([(nside//order)**2 for nside in nsides]))\n",
    "        # Number of pixels on the full sphere: 12 * nsides**2.\n",
    "\n",
    "        print('#samples per batch: {}'.format(params['batch_size']))\n",
    "#         print('=> #pixels per batch (input): {:,}'.format(params['batch_size']*(Nside//order)**2))\n",
    "#         print('=> #pixels for training (input): {:,}'.format(params['num_epochs']*ntrain*(Nside//order)**2))\n",
    "\n",
    "        n_steps = params['num_epochs'] * ntrain // params['batch_size']\n",
    "        lr = [params['scheduler'](step).eval(session=tf.Session()) for step in [0, n_steps]]\n",
    "        print('Learning rate will start at {:.1e} and finish at {:.1e}.'.format(*lr))\n",
    "\n",
    "    return params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "#sides: [1024, 512, 256, 128, 64, 32, 32]\n",
      "#samples per batch: 16\n",
      "Learning rate will start at 2.0e-04 and finish at 4.7e-05.\n",
      "build laplacian, time:  2.8578410148620605\n",
      "rescale laplacian, time:  21.98816967010498\n",
      "build laplacian, time:  22.520158767700195\n",
      "rescale laplacian, time:  24.76691198348999\n",
      "build laplacian, time:  24.957375526428223\n",
      "rescale laplacian, time:  25.347638607025146\n",
      "build laplacian, time:  25.39183497428894\n",
      "rescale laplacian, time:  25.51144003868103\n",
      "build laplacian, time:  25.522844791412354\n",
      "rescale laplacian, time:  25.560038328170776\n",
      "build laplacian, time:  25.565006256103516\n",
      "rescale laplacian, time:  25.576037883758545\n",
      "NN architecture\n",
      "  input: M_0 = 1048576\n",
      "  layer 1: cgconv1\n",
      "    representation: M_0 * F_1 / p_1 = 1048576 * 16 / 4 = 4194304\n",
      "    weights: F_0 * F_1 * K_1 = 1 * 16 * 5 = 80\n",
      "    biases: F_1 = 16\n",
      "    batch normalization\n",
      "  layer 2: cgconv2\n",
      "    representation: M_1 * F_2 / p_2 = 262144 * 32 / 4 = 2097152\n",
      "    weights: F_1 * F_2 * K_2 = 16 * 32 * 5 = 2560\n",
      "    biases: F_2 = 32\n",
      "    batch normalization\n",
      "  layer 3: cgconv3\n",
      "    representation: M_2 * F_3 / p_3 = 65536 * 64 / 4 = 1048576\n",
      "    weights: F_2 * F_3 * K_3 = 32 * 64 * 5 = 10240\n",
      "    biases: F_3 = 64\n",
      "    batch normalization\n",
      "  layer 4: cgconv4\n",
      "    representation: M_3 * F_4 / p_4 = 16384 * 64 / 4 = 262144\n",
      "    weights: F_3 * F_4 * K_4 = 64 * 64 * 5 = 20480\n",
      "    biases: F_4 = 64\n",
      "    batch normalization\n",
      "  layer 5: cgconv5\n",
      "    representation: M_4 * F_5 / p_5 = 4096 * 64 / 4 = 65536\n",
      "    weights: F_4 * F_5 * K_5 = 64 * 64 * 5 = 20480\n",
      "    biases: F_5 = 64\n",
      "    batch normalization\n",
      "  layer 6: cgconv6\n",
      "    representation: M_5 * F_6 / p_6 = 1024 * 2 / 1 = 2048\n",
      "    weights: F_5 * F_6 * K_6 = 64 * 2 * 5 = 640\n",
      "    batch normalization\n",
      "  Statistical layer: mean\n",
      "    representation: 1 * 2 = 2\n",
      "data iterator inst., time:  0.033596038818359375\n",
      "inputs, time:  0.036535024642944336\n",
      "filter0, time:  0.9619312286376953\n",
      "bn0, time:  1.0096197128295898\n",
      "relu0, time:  1.0158302783966064\n",
      "pooling0, time:  1.0193378925323486\n",
      "filter1, time:  1.246886968612671\n",
      "bn1, time:  1.329209566116333\n",
      "relu1, time:  1.3381226062774658\n",
      "pooling1, time:  1.3425240516662598\n",
      "filter2, time:  1.4148547649383545\n",
      "bn2, time:  1.463310718536377\n",
      "relu2, time:  1.4693574905395508\n",
      "pooling2, time:  1.4722998142242432\n",
      "filter3, time:  1.513610601425171\n",
      "bn3, time:  1.5613222122192383\n",
      "relu3, time:  1.5670993328094482\n",
      "pooling3, time:  1.570112705230713\n",
      "filter4, time:  1.6103322505950928\n",
      "bn4, time:  1.6570966243743896\n",
      "relu4, time:  1.6633448600769043\n",
      "pooling4, time:  1.6675159931182861\n",
      "filter5, time:  1.7075505256652832\n",
      "inference done, time:  1.7498605251312256\n",
      "loss done, time:  1.7640166282653809\n",
      "training done, time:  3.4741170406341553\n",
      "op end done, time:  3.477363348007202\n",
      "metrics done, time:  3.4951183795928955\n",
      "all done, time:  3.5433247089385986\n"
     ]
    }
   ],
   "source": [
    "params = get_params(training.N, EXP_NAME, order, Nside, ntype)\n",
    "# params['profile'] = True  # See computation time and memory usage in Tensorboard.\n",
    "# params['debug'] = True  # Debug the model in Tensorboard.\n",
    "model = models.deepsphere(**params, new=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 Find an optimal learning rate (optional)\n",
    "\n",
    "The learning rate is the most important hyper-parameter. A technique to find an optimal value is to visualize the validation loss while increasing the learning rate. One way to define the optimal learning rate is to search for the largest value looking for which the validation loss still decreases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# backup = params.copy()\n",
    "# \n",
    "# params, learning_rate = utils.test_learning_rates(params, training.N, 1e-6, 1e-1, num_epochs=20)\n",
    "# \n",
    "# shutil.rmtree('summaries/{}/'.format(params['dir_name']), ignore_errors=True)\n",
    "# shutil.rmtree('checkpoints/{}/'.format(params['dir_name']), ignore_errors=True)\n",
    "# \n",
    "# model = models.deepsphere(**params)\n",
    "# _, loss_validation, _, _ = model.fit(training, validation)\n",
    "# \n",
    "# params.update(backup)\n",
    "#\n",
    "# plt.semilogx(learning_rate, loss_validation, '.-')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.4 Training the network\n",
    "\n",
    "Here are a few remarks.\n",
    "* The model will create tensorboard summaries in the `summaries` folder. Start tensorboard with `cd summaries` then `tensorboard --logdir .`, and open <http://localhost:6006> in a browser tab to visualize training progress and statistics about the learned parameters. You can debug the model by setting `params['debug'] = True` and launching tensorboard with `tensorboard --logdir . --debugger_port 6064`.\n",
    "* You probably need a GPU to train the model in an acceptable amount of time.\n",
    "* You will get slightly different results every time the network is trained."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training from scratch\n",
      "step 14 / 1440 (epoch 0.58 / 60):\n",
      "  learning_rate = 1.97e-04, training mAP = 0.00, training loss = 6.92e-01\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/gusset/miniconda3/envs/PDMsphere/lib/python3.6/site-packages/sklearn/metrics/classification.py:1135: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  validation accuracy: 53.12 (51 / 96), f1 (weighted): 36.86, loss: 1.16e+00\n",
      "  CPU time: 57s, wall time: 70s, perf_time_load: 1.671s, perf_time: 2.775s\n",
      "step 28 / 1440 (epoch 1.17 / 60):\n",
      "  learning_rate = 1.95e-04, training mAP = 0.00, training loss = 5.98e-01\n",
      "  validation accuracy: 53.12 (51 / 96), f1 (weighted): 36.86, loss: 1.04e+00\n",
      "  CPU time: 97s, wall time: 125s, perf_time_load: 1.672s, perf_time: 2.771s\n",
      "step 42 / 1440 (epoch 1.75 / 60):\n",
      "  learning_rate = 1.92e-04, training mAP = 0.00, training loss = 6.37e-01\n",
      "  validation accuracy: 53.12 (51 / 96), f1 (weighted): 36.86, loss: 1.17e+00\n",
      "  CPU time: 130s, wall time: 173s, perf_time_load: 1.696s, perf_time: 2.683s\n",
      "step 56 / 1440 (epoch 2.33 / 60):\n",
      "  learning_rate = 1.89e-04, training mAP = 0.00, training loss = 6.18e-01\n",
      "  validation accuracy: 53.12 (51 / 96), f1 (weighted): 36.86, loss: 1.30e+00\n",
      "  CPU time: 164s, wall time: 222s, perf_time_load: 1.726s, perf_time: 2.849s\n",
      "step 70 / 1440 (epoch 2.92 / 60):\n",
      "  learning_rate = 1.87e-04, training mAP = 0.00, training loss = 5.49e-01\n",
      "  validation accuracy: 53.12 (51 / 96), f1 (weighted): 36.86, loss: 1.36e+00\n",
      "  CPU time: 197s, wall time: 269s, perf_time_load: 1.692s, perf_time: 2.856s\n",
      "step 84 / 1440 (epoch 3.50 / 60):\n",
      "  learning_rate = 1.84e-04, training mAP = 0.00, training loss = 4.58e-01\n",
      "  validation accuracy: 53.12 (51 / 96), f1 (weighted): 36.86, loss: 1.82e+00\n",
      "  CPU time: 231s, wall time: 317s, perf_time_load: 1.716s, perf_time: 2.916s\n",
      "step 98 / 1440 (epoch 4.08 / 60):\n",
      "  learning_rate = 1.82e-04, training mAP = 0.00, training loss = 4.42e-01\n",
      "  validation accuracy: 53.12 (51 / 96), f1 (weighted): 36.86, loss: 1.58e+00\n",
      "  CPU time: 264s, wall time: 366s, perf_time_load: 1.690s, perf_time: 2.905s\n",
      "step 112 / 1440 (epoch 4.67 / 60):\n",
      "  learning_rate = 1.79e-04, training mAP = 0.00, training loss = 4.61e-01\n",
      "  validation accuracy: 53.12 (51 / 96), f1 (weighted): 36.86, loss: 2.06e+00\n",
      "  CPU time: 298s, wall time: 414s, perf_time_load: 1.701s, perf_time: 2.677s\n",
      "step 126 / 1440 (epoch 5.25 / 60):\n",
      "  learning_rate = 1.76e-04, training mAP = 0.00, training loss = 3.33e-01\n",
      "  validation accuracy: 53.12 (51 / 96), f1 (weighted): 36.86, loss: 1.95e+00\n",
      "  CPU time: 331s, wall time: 462s, perf_time_load: 1.686s, perf_time: 2.798s\n",
      "step 140 / 1440 (epoch 5.83 / 60):\n",
      "  learning_rate = 1.74e-04, training mAP = 0.00, training loss = 3.44e-01\n",
      "  validation accuracy: 53.12 (51 / 96), f1 (weighted): 36.86, loss: 8.79e-01\n",
      "  CPU time: 363s, wall time: 509s, perf_time_load: 1.681s, perf_time: 2.832s\n",
      "step 154 / 1440 (epoch 6.42 / 60):\n",
      "  learning_rate = 1.72e-04, training mAP = 0.00, training loss = 3.33e-01\n",
      "  validation accuracy: 93.75 (90 / 96), f1 (weighted): 93.75, loss: 2.83e-01\n",
      "  CPU time: 395s, wall time: 556s, perf_time_load: 1.690s, perf_time: 2.797s\n",
      "step 168 / 1440 (epoch 7.00 / 60):\n",
      "  learning_rate = 1.69e-04, training mAP = 0.00, training loss = 2.60e-01\n",
      "  validation accuracy: 52.08 (50 / 96), f1 (weighted): 40.51, loss: 1.23e+00\n",
      "  CPU time: 428s, wall time: 603s, perf_time_load: 1.601s, perf_time: 2.705s\n",
      "step 182 / 1440 (epoch 7.58 / 60):\n",
      "  learning_rate = 1.67e-04, training mAP = 0.00, training loss = 3.37e-01\n",
      "  validation accuracy: 71.88 (69 / 96), f1 (weighted): 70.06, loss: 4.23e-01\n",
      "  CPU time: 462s, wall time: 652s, perf_time_load: 1.728s, perf_time: 2.886s\n",
      "step 196 / 1440 (epoch 8.17 / 60):\n",
      "  learning_rate = 1.65e-04, training mAP = 0.00, training loss = 2.18e-01\n",
      "  validation accuracy: 55.21 (53 / 96), f1 (weighted): 46.13, loss: 6.82e-01\n",
      "  CPU time: 496s, wall time: 701s, perf_time_load: 1.735s, perf_time: 2.855s\n",
      "step 210 / 1440 (epoch 8.75 / 60):\n",
      "  learning_rate = 1.62e-04, training mAP = 0.00, training loss = 1.02e-01\n",
      "  validation accuracy: 68.75 (66 / 96), f1 (weighted): 66.15, loss: 4.85e-01\n",
      "  CPU time: 531s, wall time: 750s, perf_time_load: 1.757s, perf_time: 3.001s\n",
      "step 224 / 1440 (epoch 9.33 / 60):\n",
      "  learning_rate = 1.60e-04, training mAP = 0.00, training loss = 1.65e-01\n",
      "  validation accuracy: 56.25 (54 / 96), f1 (weighted): 47.90, loss: 7.23e-01\n",
      "  CPU time: 566s, wall time: 800s, perf_time_load: 1.642s, perf_time: 2.759s\n",
      "step 238 / 1440 (epoch 9.92 / 60):\n",
      "  learning_rate = 1.58e-04, training mAP = 0.00, training loss = 1.42e-01\n",
      "  validation accuracy: 46.88 (45 / 96), f1 (weighted): 29.92, loss: 1.82e+00\n",
      "  CPU time: 600s, wall time: 849s, perf_time_load: 1.775s, perf_time: 2.993s\n",
      "step 252 / 1440 (epoch 10.50 / 60):\n",
      "  learning_rate = 1.56e-04, training mAP = 0.00, training loss = 1.23e-01\n",
      "  validation accuracy: 46.88 (45 / 96), f1 (weighted): 29.92, loss: 2.81e+00\n",
      "  CPU time: 635s, wall time: 899s, perf_time_load: 1.785s, perf_time: 2.959s\n",
      "step 266 / 1440 (epoch 11.08 / 60):\n",
      "  learning_rate = 1.53e-04, training mAP = 0.00, training loss = 1.01e-01\n",
      "  validation accuracy: 46.88 (45 / 96), f1 (weighted): 29.92, loss: 1.82e+00\n",
      "  CPU time: 669s, wall time: 947s, perf_time_load: 1.593s, perf_time: 2.705s\n",
      "step 280 / 1440 (epoch 11.67 / 60):\n",
      "  learning_rate = 1.51e-04, training mAP = 0.00, training loss = 1.39e-01\n",
      "  validation accuracy: 78.12 (75 / 96), f1 (weighted): 77.36, loss: 3.33e-01\n",
      "  CPU time: 705s, wall time: 998s, perf_time_load: 1.840s, perf_time: 3.351s\n"
     ]
    }
   ],
   "source": [
    "accuracy_validation, loss_validation, loss_training, t_step, t_batch = model.fit(training, validation)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# npix = 12*1024**2\n",
    "# npixel = [npix/(12*o**2) for o in range(1,5)]\n",
    "# npixel = [npix]+npixel\n",
    "# npixel = npixel+[12*32**2]\n",
    "# DeepSphere = [201, 175, 42, 10.3, 2.8, 1.0]\n",
    "# plt.loglog(npixel, DeepSphere, label='DeepSphere')\n",
    "# npixel_cohen = [4*bw**2 for bw in [64, 128, 220]]\n",
    "# t_cohen = [12, 32, 175]\n",
    "# t_esteves = [9, 16]\n",
    "# plt.loglog(npixel_cohen, t_cohen, label='Cohen')\n",
    "# plt.loglog(npixel_cohen[:-1], t_esteves, label='Esteves')\n",
    "# plt.legend(fontsize=12)\n",
    "# plt.xlabel('number of pixels', fontsize=15)\n",
    "# plt.ylabel('Inference speed [s]', fontsize = 15)\n",
    "# plt.xticks(fontsize=12)\n",
    "# plt.yticks(fontsize=12)\n",
    "# plt.savefig('./figures/cosmo_time.png', bboxes_inches='tight')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see below that the classifier does not overfit the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot.plot_loss(loss_training, loss_validation, t_step, params['eval_frequency'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "error_validation = experiment_helper.model_error(model, features_validation[:,:,np.newaxis], labels_validation)\n",
    "print('The validation error is {:.2%}'.format(error_validation), flush=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "error_test = experiment_helper.model_error(model, features_test[:,:,np.newaxis], labels_test)\n",
    "print('The testing error is {:.2%}'.format(error_test), flush=True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
